Step 1: Load the data¶

In [1]:
!pip install pandas

Requirement already satisfied: pandas in c:\programdata\anaconda3\lib\site-packages (1.5.3)
Requirement already satisfied: python-dateutil>=2.8.1 in c:\programdata\anaconda3\lib\site-packages (from pandas) (2.8.2)
Requirement already satisfied: numpy>=1.20.3 in c:\programdata\anaconda3\lib\site-packages (from pandas) (1.23.5)
Requirement already satisfied: pytz>=2020.1 in c:\programdata\anaconda3\lib\site-packages (from pandas) (2022.7)
Requirement already satisfied: six>=1.5 in c:\programdata\anaconda3\lib\site-packages (from python-dateutil>=2.8.1->pandas) (1.16.0)

[notice] A new release of pip available: 22.3.1 -> 23.0.1
[notice] To update, run: python.exe -m pip install --upgrade pip
In [2]:
import pandas as pd
import numpy as np

df = pd.read_csv("star_wars_character_dataset.csv")
In [3]:
df.head()
Out[3]:
name height mass hair_color skin_color eye_color birth_year sex gender homeworld species films vehicles starships
0 Luke Skywalker 172.0 77.0 blond fair blue 19.0 male masculine Tatooine Human The Empire Strikes Back, Revenge of the Sith, ... Snowspeeder, Imperial Speeder Bike X-wing, Imperial shuttle
1 C-3PO 167.0 75.0 NaN gold yellow 112.0 none masculine Tatooine Droid The Empire Strikes Back, Attack of the Clones,... NaN NaN
2 R2-D2 96.0 32.0 NaN white, blue red 33.0 none masculine Naboo Droid The Empire Strikes Back, Attack of the Clones,... NaN NaN
3 Darth Vader 202.0 136.0 none white yellow 41.9 male masculine Tatooine Human The Empire Strikes Back, Revenge of the Sith, ... NaN TIE Advanced x1
4 Leia Organa 150.0 49.0 brown light brown 19.0 female feminine Alderaan Human The Empire Strikes Back, Revenge of the Sith, ... Imperial Speeder Bike NaN
In [4]:
df.shape
Out[4]:
(87, 14)
In [5]:
df.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 87 entries, 0 to 86
Data columns (total 14 columns):
 #   Column      Non-Null Count  Dtype  
---  ------      --------------  -----  
 0   name        87 non-null     object 
 1   height      81 non-null     float64
 2   mass        59 non-null     float64
 3   hair_color  82 non-null     object 
 4   skin_color  87 non-null     object 
 5   eye_color   87 non-null     object 
 6   birth_year  43 non-null     float64
 7   sex         83 non-null     object 
 8   gender      83 non-null     object 
 9   homeworld   77 non-null     object 
 10  species     83 non-null     object 
 11  films       87 non-null     object 
 12  vehicles    11 non-null     object 
 13  starships   20 non-null     object 
dtypes: float64(3), object(11)
memory usage: 9.6+ KB
In [6]:
df.isnull().sum()
Out[6]:
name           0
height         6
mass          28
hair_color     5
skin_color     0
eye_color      0
birth_year    44
sex            4
gender         4
homeworld     10
species        4
films          0
vehicles      76
starships     67
dtype: int64
In [7]:
df.mass.describe()
Out[7]:
count      59.000000
mean       97.311864
std       169.457163
min        15.000000
25%        55.600000
50%        79.000000
75%        84.500000
max      1358.000000
Name: mass, dtype: float64
In [8]:
## removing max mass to adjust the mean and std and filling null values with mean data
df.drop(df[df.mass == np.max(df.mass)].index,inplace=True)
df.mass.fillna(df.mass.mean(),inplace=True)
In [9]:
df.birth_year.describe()
Out[9]:
count     42.000000
mean      75.364286
std      133.999900
min        8.000000
25%       34.000000
50%       50.000000
75%       70.750000
max      896.000000
Name: birth_year, dtype: float64
In [10]:
## removing max birth_year to adjust the mean and std and filling null values with mean data
df.drop(df[df.birth_year == np.max(df.birth_year)].index,inplace=True)
df.drop(df[df.birth_year == np.max(df.birth_year)].index,inplace=True)
df.birth_year.fillna(df.birth_year.mean(),inplace=True)
In [11]:
df.birth_year.describe()
Out[11]:
count     84.000000
mean      51.732500
std       17.313345
min        8.000000
25%       50.799375
50%       51.732500
75%       51.732500
max      112.000000
Name: birth_year, dtype: float64
In [12]:
df["hair_color"] = df.hair_color.str.split(',').str[0]
df["eye_color"] = df.eye_color.str.split(',').str[0]
df["skin_color"] = df.skin_color.str.split(',').str[0]
In [13]:
df["hair_color"].value_counts()
Out[13]:
none       37
brown      18
black      13
blond       3
auburn      3
white       3
grey        1
blonde      1
unknown     1
Name: hair_color, dtype: int64
In [14]:
## filling null values for hair color with none 
df.hair_color.fillna('none',inplace=True)
In [15]:
df.species.value_counts()
Out[15]:
Human           35
Droid            6
Gungan           3
Mirialan         2
Twi'lek          2
Zabrak           2
Kaminoan         2
Aleena           1
Skakoan          1
Quermian         1
Besalisk         1
Muun             1
Togruta          1
Clawdite         1
Kaleesh          1
Geonosian        1
Chagrian         1
Wookiee          1
Kel Dor          1
Nautolan         1
Iktotchi         1
Tholothian       1
Cerean           1
Toong            1
Xexto            1
Vulptereen       1
Dug              1
Toydarian        1
Neimodian        1
Sullustan        1
Ewok             1
Mon Calamari     1
Trandoshan       1
Rodian           1
Pau'an           1
Name: species, dtype: int64
In [16]:
df.homeworld.value_counts().count()
Out[16]:
47
In [17]:
## filling null values for sex, gender, homeworld and species with random 
import random
df.gender.fillna(random.choice(['masculine','feminine']),inplace=True)
df.sex.fillna(random.choice(['female','male','hermaphroditic']),inplace=True)
df.homeworld.fillna(random.choice(['Naboo','Tatooine']),inplace=True)
df.species.fillna(random.choice(['Human','Droid']),inplace=True)
In [18]:
df.info()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 84 entries, 0 to 86
Data columns (total 14 columns):
 #   Column      Non-Null Count  Dtype  
---  ------      --------------  -----  
 0   name        84 non-null     object 
 1   height      78 non-null     float64
 2   mass        84 non-null     float64
 3   hair_color  84 non-null     object 
 4   skin_color  84 non-null     object 
 5   eye_color   84 non-null     object 
 6   birth_year  84 non-null     float64
 7   sex         84 non-null     object 
 8   gender      84 non-null     object 
 9   homeworld   84 non-null     object 
 10  species     84 non-null     object 
 11  films       84 non-null     object 
 12  vehicles    10 non-null     object 
 13  starships   19 non-null     object 
dtypes: float64(3), object(11)
memory usage: 9.8+ KB
In [19]:
## removing vehicles and starships as it's mostly empty and filling mean height for remaining dataframe
df.drop(['vehicles','starships'],axis=1,inplace=True)
df.height.fillna(df.height.mean(),inplace=True)
In [20]:
## no duplicate values
df.duplicated().sum()
Out[20]:
0
In [21]:
df
Out[21]:
name height mass hair_color skin_color eye_color birth_year sex gender homeworld species films
0 Luke Skywalker 172.000000 77.000000 blond fair blue 19.0000 male masculine Tatooine Human The Empire Strikes Back, Revenge of the Sith, ...
1 C-3PO 167.000000 75.000000 none gold yellow 112.0000 none masculine Tatooine Droid The Empire Strikes Back, Attack of the Clones,...
2 R2-D2 96.000000 32.000000 none white red 33.0000 none masculine Naboo Droid The Empire Strikes Back, Attack of the Clones,...
3 Darth Vader 202.000000 136.000000 none white yellow 41.9000 male masculine Tatooine Human The Empire Strikes Back, Revenge of the Sith, ...
4 Leia Organa 150.000000 49.000000 brown light brown 19.0000 female feminine Alderaan Human The Empire Strikes Back, Revenge of the Sith, ...
... ... ... ... ... ... ... ... ... ... ... ... ...
82 Rey 175.051282 75.575862 brown light hazel 51.7325 female feminine Naboo Human The Force Awakens
83 Poe Dameron 175.051282 75.575862 brown light brown 51.7325 male masculine Naboo Human The Force Awakens
84 BB8 175.051282 75.575862 none none black 51.7325 none masculine Naboo Droid The Force Awakens
85 Captain Phasma 175.051282 75.575862 unknown unknown unknown 51.7325 hermaphroditic masculine Naboo Human The Force Awakens
86 Padmé Amidala 165.000000 45.000000 brown light brown 46.0000 female feminine Naboo Human Attack of the Clones, The Phantom Menace, Reve...

84 rows × 12 columns

Step 2: Data Preprocessing¶

In [22]:
df_object = df[['hair_color','skin_color','eye_color','sex','gender','species','films']]

df_float = df[['height','mass','birth_year']]
In [23]:
# Training the Model
from sklearn.linear_model import LogisticRegression
In [24]:
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OrdinalEncoder, OneHotEncoder
from sklearn.pipeline import make_pipeline

pipe = make_pipeline(
    ColumnTransformer(
        transformers=[
            ("encode", OneHotEncoder(), ["hair_color",	"skin_color",	"eye_color", "sex",	"gender",	"species",	"films"]),
        ],
        remainder="passthrough",
    ),
)
In [25]:
enco = pipe.fit_transform(df_object).toarray()
In [26]:
enco.shape, df_float.shape
Out[26]:
((84, 107), (84, 3))
In [27]:
frames =  [enco, df_float.values]
In [28]:
X = np.concatenate(frames, axis = 1)
In [29]:
X
Out[29]:
array([[  0.        ,   0.        ,   1.        , ..., 172.        ,
         77.        ,  19.        ],
       [  0.        ,   0.        ,   0.        , ..., 167.        ,
         75.        , 112.        ],
       [  0.        ,   0.        ,   0.        , ...,  96.        ,
         32.        ,  33.        ],
       ...,
       [  0.        ,   0.        ,   0.        , ..., 175.05128205,
         75.57586207,  51.7325    ],
       [  0.        ,   0.        ,   0.        , ..., 175.05128205,
         75.57586207,  51.7325    ],
       [  0.        ,   0.        ,   0.        , ..., 165.        ,
         45.        ,  46.        ]])
In [30]:
ordinalencoder = OrdinalEncoder()
y = (ordinalencoder.fit_transform(df.homeworld.values.reshape(-1,1)))
In [31]:
y.shape
Out[31]:
(84, 1)
In [32]:
ordinalencoder.categories_
Out[32]:
[array(['Alderaan', 'Aleen Minor', 'Bespin', 'Bestine IV',
        'Cato Neimoidia', 'Cerea', 'Champala', 'Chandrila', 'Concord Dawn',
        'Corellia', 'Coruscant', 'Dathomir', 'Dorin', 'Endor', 'Eriadu',
        'Geonosis', 'Glee Anselm', 'Haruun Kal', 'Iktotch', 'Iridonia',
        'Kalee', 'Kamino', 'Kashyyyk', 'Malastare', 'Mirial', 'Mon Cala',
        'Muunilinst', 'Naboo', 'Ojom', 'Quermia', 'Rodia', 'Ryloth',
        'Serenno', 'Shili', 'Skako', 'Socorro', 'Stewjon', 'Sullust',
        'Tatooine', 'Toydaria', 'Trandosha', 'Troiken', 'Tund', 'Umbara',
        'Utapau', 'Vulpter', 'Zolan'], dtype=object)]
In [33]:
for i, n in enumerate(ordinalencoder.categories_[0]):
    print(i, n)

0 Alderaan
1 Aleen Minor
2 Bespin
3 Bestine IV
4 Cato Neimoidia
5 Cerea
6 Champala
7 Chandrila
8 Concord Dawn
9 Corellia
10 Coruscant
11 Dathomir
12 Dorin
13 Endor
14 Eriadu
15 Geonosis
16 Glee Anselm
17 Haruun Kal
18 Iktotch
19 Iridonia
20 Kalee
21 Kamino
22 Kashyyyk
23 Malastare
24 Mirial
25 Mon Cala
26 Muunilinst
27 Naboo
28 Ojom
29 Quermia
30 Rodia
31 Ryloth
32 Serenno
33 Shili
34 Skako
35 Socorro
36 Stewjon
37 Sullust
38 Tatooine
39 Toydaria
40 Trandosha
41 Troiken
42 Tund
43 Umbara
44 Utapau
45 Vulpter
46 Zolan
In [34]:
from imblearn.over_sampling import RandomOverSampler
rus = RandomOverSampler(random_state=0)
rus.fit(X, y)
X_train_smote, y_train_smote = rus.fit_resample(X, y)
In [35]:
X_train_smote.shape, y_train_smote.shape
Out[35]:
((940, 110), (940,))
In [36]:
# Split the Data into train and test
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(X_train_smote,y_train_smote,test_size=0.3,random_state=42)
In [37]:
#shapes of splitted data
print("X_train:",X_train.shape)
print("X_test:",X_test.shape)
print("Y_train:",y_train.shape)
print("Y_test:",y_test.shape)

X_train: (658, 110)
X_test: (282, 110)
Y_train: (658,)
Y_test: (282,)

Step 3: Creating Baseline¶

In [38]:
logi = LogisticRegression()
In [39]:
baseline = logi.fit(X_train, y_train)

C:\Users\riyup\.conda\envs\tf_venv\lib\site-packages\sklearn\linear_model\_logistic.py:458: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
    https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
  n_iter_i = _check_optimize_result(
In [40]:
y_pred = baseline.predict(X_test)
In [41]:
from sklearn.metrics import r2_score, accuracy_score, f1_score, mean_squared_error, confusion_matrix

r2_score_value = r2_score(y_test, y_pred)
r2_score_value
Out[41]:
0.7601965426922674
In [42]:
accuracy_score(y_test, y_pred)
Out[42]:
0.8829787234042553
In [ ]:
In [43]:
import seaborn as sns
sns.regplot(x=y_test, y=y_pred, ci=None, scatter_kws={"color": "black"}, line_kws={"color": "red"});
In [44]:
import math
mse_baseline = mean_squared_error(y_test, y_pred)  
rmse_baseline = math.sqrt(mse_baseline)  
  
print("Root Mean Squared Error: ", rmse_baseline) 
print("Mean Squared Error: ", mse_baseline)

Root Mean Squared Error:  6.4667726831991095
Mean Squared Error:  41.819148936170215

Model 1: With 2 Dense layers¶

In [45]:
import tensorflow as tf
from tensorflow.keras import layers, optimizers
In [46]:
X_train_tf = tf.convert_to_tensor(X_train.astype(np.float64))
X_test_tf = tf.convert_to_tensor(X_test.astype(np.float64))
y_train_tf = tf.convert_to_tensor(y_train.astype(np.float64))
y_test_tf = tf.convert_to_tensor(y_test.astype(np.float64))
In [47]:
X_train.shape, X_test.shape
Out[47]:
((658, 110), (282, 110))
In [48]:
X_test_tf.shape, X_train_tf.shape
Out[48]:
(TensorShape([282, 110]), TensorShape([658, 110]))
In [49]:
import matplotlib.pyplot as plt
def plot_loss_curves(history):
    """
    Returns separate loss curves for training and validation metrics.

    Args:
      history: TensorFlow model History object (see: https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/History)
    """
    loss = history.history['loss']
    val_loss = history.history['val_loss']

    accuracy = history.history['accuracy']
    val_accuracy = history.history['val_accuracy']

    epochs = range(len(history.history['loss']))

    # Plot accuracy
    plt.figure()
    plt.plot(epochs, loss, label='train_loss')
    plt.plot(epochs, val_loss, label='val_loss')
    plt.title('Loss')
    plt.xlabel('Epochs')
    plt.legend();

    # Plot accuracy
    plt.figure()
    plt.plot(epochs, accuracy, label='train_accuracy')
    plt.plot(epochs, val_accuracy, label='val_accuracy')
    plt.title('Accuracy')
    plt.xlabel('Epochs')
    plt.legend();
In [50]:
model_1 = tf.keras.Sequential([
    layers.Input(shape=(X_train.shape[1],)),
    layers.Dense(256, activation="relu"),
    layers.Dense(47, activation="softmax")
])

model_1.compile(loss="sparse_categorical_crossentropy", optimizer=optimizers.Adam(), metrics=["accuracy"])
In [51]:
history_1 = model_1.fit(X_train_tf, y_train_tf, epochs=100, validation_data=(X_test, y_test))

Epoch 1/100
21/21 [==============================] - 1s 11ms/step - loss: 11.3743 - accuracy: 0.0289 - val_loss: 5.4935 - val_accuracy: 0.0355
Epoch 2/100
21/21 [==============================] - 0s 4ms/step - loss: 4.5734 - accuracy: 0.0836 - val_loss: 3.5204 - val_accuracy: 0.1383
Epoch 3/100
21/21 [==============================] - 0s 4ms/step - loss: 3.0919 - accuracy: 0.2204 - val_loss: 2.7962 - val_accuracy: 0.3227
Epoch 4/100
21/21 [==============================] - 0s 5ms/step - loss: 2.3759 - accuracy: 0.4438 - val_loss: 2.2907 - val_accuracy: 0.5390
Epoch 5/100
21/21 [==============================] - 0s 6ms/step - loss: 1.9371 - accuracy: 0.5502 - val_loss: 1.9377 - val_accuracy: 0.6170
Epoch 6/100
21/21 [==============================] - 0s 5ms/step - loss: 1.5749 - accuracy: 0.7249 - val_loss: 1.6013 - val_accuracy: 0.6879
Epoch 7/100
21/21 [==============================] - 0s 4ms/step - loss: 1.3267 - accuracy: 0.7872 - val_loss: 1.3394 - val_accuracy: 0.7837
Epoch 8/100
21/21 [==============================] - 0s 5ms/step - loss: 1.1487 - accuracy: 0.8161 - val_loss: 1.0488 - val_accuracy: 0.9220
Epoch 9/100
21/21 [==============================] - 0s 3ms/step - loss: 0.9799 - accuracy: 0.8556 - val_loss: 0.9168 - val_accuracy: 0.8972
Epoch 10/100
21/21 [==============================] - 0s 3ms/step - loss: 0.8312 - accuracy: 0.9073 - val_loss: 0.8775 - val_accuracy: 0.8546
Epoch 11/100
21/21 [==============================] - 0s 3ms/step - loss: 0.7005 - accuracy: 0.9088 - val_loss: 0.7926 - val_accuracy: 0.8794
Epoch 12/100
21/21 [==============================] - 0s 3ms/step - loss: 0.5750 - accuracy: 0.9392 - val_loss: 0.6272 - val_accuracy: 0.8830
Epoch 13/100
21/21 [==============================] - 0s 3ms/step - loss: 0.5058 - accuracy: 0.9514 - val_loss: 0.5460 - val_accuracy: 0.9504
Epoch 14/100
21/21 [==============================] - 0s 3ms/step - loss: 0.4139 - accuracy: 0.9574 - val_loss: 0.4873 - val_accuracy: 0.9397
Epoch 15/100
21/21 [==============================] - 0s 3ms/step - loss: 0.3812 - accuracy: 0.9620 - val_loss: 0.4014 - val_accuracy: 0.9468
Epoch 16/100
21/21 [==============================] - 0s 4ms/step - loss: 0.3299 - accuracy: 0.9681 - val_loss: 0.3380 - val_accuracy: 0.9610
Epoch 17/100
21/21 [==============================] - 0s 3ms/step - loss: 0.2963 - accuracy: 0.9711 - val_loss: 0.3254 - val_accuracy: 0.9610
Epoch 18/100
21/21 [==============================] - 0s 3ms/step - loss: 0.2547 - accuracy: 0.9757 - val_loss: 0.2995 - val_accuracy: 0.9645
Epoch 19/100
21/21 [==============================] - 0s 3ms/step - loss: 0.2407 - accuracy: 0.9726 - val_loss: 0.2841 - val_accuracy: 0.9504
Epoch 20/100
21/21 [==============================] - 0s 3ms/step - loss: 0.2113 - accuracy: 0.9787 - val_loss: 0.2558 - val_accuracy: 0.9645
Epoch 21/100
21/21 [==============================] - 0s 3ms/step - loss: 0.1837 - accuracy: 0.9878 - val_loss: 0.2342 - val_accuracy: 0.9645
Epoch 22/100
21/21 [==============================] - 0s 4ms/step - loss: 0.1701 - accuracy: 0.9802 - val_loss: 0.1885 - val_accuracy: 0.9787
Epoch 23/100
21/21 [==============================] - 0s 4ms/step - loss: 0.1496 - accuracy: 0.9863 - val_loss: 0.2241 - val_accuracy: 0.9645
Epoch 24/100
21/21 [==============================] - 0s 4ms/step - loss: 0.1523 - accuracy: 0.9818 - val_loss: 0.2642 - val_accuracy: 0.9574
Epoch 25/100
21/21 [==============================] - 0s 4ms/step - loss: 0.1297 - accuracy: 0.9878 - val_loss: 0.1752 - val_accuracy: 0.9752
Epoch 26/100
21/21 [==============================] - 0s 4ms/step - loss: 0.1234 - accuracy: 0.9894 - val_loss: 0.2092 - val_accuracy: 0.9610
Epoch 27/100
21/21 [==============================] - 0s 3ms/step - loss: 0.1310 - accuracy: 0.9833 - val_loss: 0.1530 - val_accuracy: 0.9858
Epoch 28/100
21/21 [==============================] - 0s 3ms/step - loss: 0.1055 - accuracy: 0.9894 - val_loss: 0.1248 - val_accuracy: 0.9929
Epoch 29/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0897 - accuracy: 0.9939 - val_loss: 0.1294 - val_accuracy: 0.9965
Epoch 30/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0954 - accuracy: 0.9863 - val_loss: 0.1403 - val_accuracy: 0.9894
Epoch 31/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0979 - accuracy: 0.9909 - val_loss: 0.1130 - val_accuracy: 0.9965
Epoch 32/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0929 - accuracy: 0.9924 - val_loss: 0.1379 - val_accuracy: 0.9858
Epoch 33/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0828 - accuracy: 0.9924 - val_loss: 0.1263 - val_accuracy: 0.9823
Epoch 34/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0744 - accuracy: 0.9924 - val_loss: 0.0969 - val_accuracy: 0.9894
Epoch 35/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0656 - accuracy: 0.9954 - val_loss: 0.0934 - val_accuracy: 0.9965
Epoch 36/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0611 - accuracy: 0.9954 - val_loss: 0.1087 - val_accuracy: 0.9823
Epoch 37/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0617 - accuracy: 0.9939 - val_loss: 0.1059 - val_accuracy: 0.9823
Epoch 38/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0665 - accuracy: 0.9924 - val_loss: 0.0917 - val_accuracy: 0.9894
Epoch 39/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0656 - accuracy: 0.9894 - val_loss: 0.0861 - val_accuracy: 0.9894
Epoch 40/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0500 - accuracy: 0.9970 - val_loss: 0.0759 - val_accuracy: 0.9965
Epoch 41/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0448 - accuracy: 0.9970 - val_loss: 0.0829 - val_accuracy: 0.9894
Epoch 42/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0460 - accuracy: 0.9970 - val_loss: 0.0790 - val_accuracy: 0.9929
Epoch 43/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0408 - accuracy: 0.9954 - val_loss: 0.0802 - val_accuracy: 0.9894
Epoch 44/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0394 - accuracy: 0.9970 - val_loss: 0.0794 - val_accuracy: 0.9894
Epoch 45/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0387 - accuracy: 0.9970 - val_loss: 0.0749 - val_accuracy: 0.9823
Epoch 46/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0342 - accuracy: 1.0000 - val_loss: 0.0720 - val_accuracy: 0.9894
Epoch 47/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0350 - accuracy: 0.9954 - val_loss: 0.0638 - val_accuracy: 0.9965
Epoch 48/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0345 - accuracy: 0.9970 - val_loss: 0.0606 - val_accuracy: 0.9965
Epoch 49/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0364 - accuracy: 0.9970 - val_loss: 0.0696 - val_accuracy: 0.9929
Epoch 50/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0309 - accuracy: 0.9985 - val_loss: 0.0834 - val_accuracy: 0.9787
Epoch 51/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0293 - accuracy: 0.9985 - val_loss: 0.0670 - val_accuracy: 0.9894
Epoch 52/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0264 - accuracy: 1.0000 - val_loss: 0.0648 - val_accuracy: 0.9894
Epoch 53/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0277 - accuracy: 0.9985 - val_loss: 0.0562 - val_accuracy: 0.9965
Epoch 54/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0319 - accuracy: 0.9970 - val_loss: 0.0709 - val_accuracy: 0.9858
Epoch 55/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0309 - accuracy: 0.9970 - val_loss: 0.0694 - val_accuracy: 0.9823
Epoch 56/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0243 - accuracy: 0.9985 - val_loss: 0.0523 - val_accuracy: 0.9965
Epoch 57/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0228 - accuracy: 1.0000 - val_loss: 0.0661 - val_accuracy: 0.9858
Epoch 58/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0280 - accuracy: 0.9985 - val_loss: 0.0714 - val_accuracy: 0.9823
Epoch 59/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0210 - accuracy: 1.0000 - val_loss: 0.0563 - val_accuracy: 0.9894
Epoch 60/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0198 - accuracy: 1.0000 - val_loss: 0.0526 - val_accuracy: 0.9965
Epoch 61/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0207 - accuracy: 0.9985 - val_loss: 0.0584 - val_accuracy: 0.9894
Epoch 62/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0182 - accuracy: 1.0000 - val_loss: 0.0503 - val_accuracy: 0.9965
Epoch 63/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0168 - accuracy: 1.0000 - val_loss: 0.0476 - val_accuracy: 0.9965
Epoch 64/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0167 - accuracy: 0.9985 - val_loss: 0.0520 - val_accuracy: 0.9929
Epoch 65/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0157 - accuracy: 1.0000 - val_loss: 0.0519 - val_accuracy: 0.9965
Epoch 66/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0162 - accuracy: 1.0000 - val_loss: 0.0465 - val_accuracy: 0.9965
Epoch 67/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0153 - accuracy: 1.0000 - val_loss: 0.0465 - val_accuracy: 0.9965
Epoch 68/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0142 - accuracy: 1.0000 - val_loss: 0.0526 - val_accuracy: 0.9929
Epoch 69/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0174 - accuracy: 0.9985 - val_loss: 0.0583 - val_accuracy: 0.9823
Epoch 70/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0219 - accuracy: 0.9970 - val_loss: 0.0635 - val_accuracy: 0.9787
Epoch 71/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0152 - accuracy: 1.0000 - val_loss: 0.0457 - val_accuracy: 0.9965
Epoch 72/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0129 - accuracy: 1.0000 - val_loss: 0.0571 - val_accuracy: 0.9823
Epoch 73/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0134 - accuracy: 1.0000 - val_loss: 0.0465 - val_accuracy: 0.9965
Epoch 74/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0128 - accuracy: 0.9985 - val_loss: 0.0444 - val_accuracy: 0.9965
Epoch 75/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0111 - accuracy: 1.0000 - val_loss: 0.0497 - val_accuracy: 0.9965
Epoch 76/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0120 - accuracy: 1.0000 - val_loss: 0.0460 - val_accuracy: 0.9965
Epoch 77/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0106 - accuracy: 1.0000 - val_loss: 0.0433 - val_accuracy: 0.9965
Epoch 78/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0107 - accuracy: 1.0000 - val_loss: 0.0439 - val_accuracy: 0.9965
Epoch 79/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0110 - accuracy: 1.0000 - val_loss: 0.0473 - val_accuracy: 0.9858
Epoch 80/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0110 - accuracy: 1.0000 - val_loss: 0.0431 - val_accuracy: 0.9965
Epoch 81/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0109 - accuracy: 1.0000 - val_loss: 0.0440 - val_accuracy: 0.9965
Epoch 82/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0114 - accuracy: 1.0000 - val_loss: 0.0444 - val_accuracy: 0.9965
Epoch 83/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0117 - accuracy: 0.9985 - val_loss: 0.0419 - val_accuracy: 0.9929
Epoch 84/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0087 - accuracy: 1.0000 - val_loss: 0.0494 - val_accuracy: 0.9965
Epoch 85/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0092 - accuracy: 1.0000 - val_loss: 0.0424 - val_accuracy: 0.9965
Epoch 86/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0083 - accuracy: 1.0000 - val_loss: 0.0454 - val_accuracy: 0.9894
Epoch 87/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0081 - accuracy: 1.0000 - val_loss: 0.0477 - val_accuracy: 0.9965
Epoch 88/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0079 - accuracy: 1.0000 - val_loss: 0.0420 - val_accuracy: 0.9965
Epoch 89/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0079 - accuracy: 1.0000 - val_loss: 0.0422 - val_accuracy: 0.9965
Epoch 90/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0077 - accuracy: 1.0000 - val_loss: 0.0453 - val_accuracy: 0.9965
Epoch 91/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0070 - accuracy: 1.0000 - val_loss: 0.0411 - val_accuracy: 0.9965
Epoch 92/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0072 - accuracy: 1.0000 - val_loss: 0.0419 - val_accuracy: 0.9965
Epoch 93/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0074 - accuracy: 1.0000 - val_loss: 0.0456 - val_accuracy: 0.9965
Epoch 94/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0069 - accuracy: 1.0000 - val_loss: 0.0415 - val_accuracy: 0.9965
Epoch 95/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0066 - accuracy: 1.0000 - val_loss: 0.0436 - val_accuracy: 0.9965
Epoch 96/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0065 - accuracy: 1.0000 - val_loss: 0.0456 - val_accuracy: 0.9965
Epoch 97/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0069 - accuracy: 1.0000 - val_loss: 0.0414 - val_accuracy: 0.9965
Epoch 98/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0063 - accuracy: 1.0000 - val_loss: 0.0418 - val_accuracy: 0.9965
Epoch 99/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0058 - accuracy: 1.0000 - val_loss: 0.0469 - val_accuracy: 0.9894
Epoch 100/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0063 - accuracy: 1.0000 - val_loss: 0.0419 - val_accuracy: 0.9965
In [52]:
y_pred_1 = model_1.predict(X_test_tf)

9/9 [==============================] - 0s 1ms/step
In [53]:
model_1.evaluate(X_test_tf, y_test_tf)

9/9 [==============================] - 0s 982us/step - loss: 0.0419 - accuracy: 0.9965
Out[53]:
[0.04191071540117264, 0.9964538812637329]
In [54]:
df_history_1 = pd.DataFrame(history_1.history)
df_history_1
Out[54]:
loss accuracy val_loss val_accuracy
0 11.374345 0.028875 5.493479 0.035461
1 4.573410 0.083587 3.520423 0.138298
2 3.091890 0.220365 2.796206 0.322695
3 2.375943 0.443769 2.290737 0.539007
4 1.937063 0.550152 1.937668 0.617021
... ... ... ... ...
95 0.006547 1.000000 0.045550 0.996454
96 0.006922 1.000000 0.041447 0.996454
97 0.006258 1.000000 0.041751 0.996454
98 0.005798 1.000000 0.046906 0.989362
99 0.006344 1.000000 0.041911 0.996454

100 rows × 4 columns

In [55]:
import plotly.express as px

fig = px.line(df_history_1, y=['loss', 'val_loss'], labels=['Loss', 'Validation Loss'])
fig.show()

Model 2: With SGD Optimizer¶

In [56]:
model_2 = tf.keras.Sequential([
    layers.Input(shape=(X_train.shape[1],)),
    layers.Dense(256, activation="relu"),
    layers.Dense(47, activation="softmax")
])

model_2.compile(loss="sparse_categorical_crossentropy", optimizer=optimizers.SGD(), metrics=["accuracy"])
In [57]:
history_2 = model_2.fit(X_train_tf, y_train_tf, epochs=100, validation_data=(X_test, y_test))

Epoch 1/100
21/21 [==============================] - 0s 10ms/step - loss: 20.4059 - accuracy: 0.0152 - val_loss: 3.8965 - val_accuracy: 0.0071
Epoch 2/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8435 - accuracy: 0.0198 - val_loss: 3.8382 - val_accuracy: 0.0035
Epoch 3/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8376 - accuracy: 0.0289 - val_loss: 3.8410 - val_accuracy: 0.0213
Epoch 4/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8370 - accuracy: 0.0456 - val_loss: 3.8402 - val_accuracy: 0.0213
Epoch 5/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8336 - accuracy: 0.0441 - val_loss: 3.8607 - val_accuracy: 0.0177
Epoch 6/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8314 - accuracy: 0.0441 - val_loss: 3.8511 - val_accuracy: 0.0177
Epoch 7/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8232 - accuracy: 0.0486 - val_loss: 3.8304 - val_accuracy: 0.0213
Epoch 8/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8177 - accuracy: 0.0441 - val_loss: 3.8261 - val_accuracy: 0.0213
Epoch 9/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8183 - accuracy: 0.0426 - val_loss: 3.8249 - val_accuracy: 0.0213
Epoch 10/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8077 - accuracy: 0.0456 - val_loss: 3.8186 - val_accuracy: 0.0213
Epoch 11/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8067 - accuracy: 0.0334 - val_loss: 3.8415 - val_accuracy: 0.0000e+00
Epoch 12/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8480 - accuracy: 0.0274 - val_loss: 3.8520 - val_accuracy: 0.0035
Epoch 13/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8446 - accuracy: 0.0289 - val_loss: 3.8468 - val_accuracy: 0.0035
Epoch 14/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8440 - accuracy: 0.0228 - val_loss: 3.8516 - val_accuracy: 0.0035
Epoch 15/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8418 - accuracy: 0.0213 - val_loss: 3.8285 - val_accuracy: 0.0000e+00
Epoch 16/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8416 - accuracy: 0.0106 - val_loss: 3.8403 - val_accuracy: 0.0035
Epoch 17/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8381 - accuracy: 0.0274 - val_loss: 3.8376 - val_accuracy: 0.0213
Epoch 18/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8310 - accuracy: 0.0380 - val_loss: 3.8154 - val_accuracy: 0.0177
Epoch 19/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8286 - accuracy: 0.0365 - val_loss: 3.8423 - val_accuracy: 0.0213
Epoch 20/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8285 - accuracy: 0.0456 - val_loss: 3.8414 - val_accuracy: 0.0213
Epoch 21/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8163 - accuracy: 0.0274 - val_loss: 3.8353 - val_accuracy: 0.0213
Epoch 22/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8191 - accuracy: 0.0395 - val_loss: 3.8366 - val_accuracy: 0.0213
Epoch 23/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8036 - accuracy: 0.0350 - val_loss: 3.8184 - val_accuracy: 0.0213
Epoch 24/100
21/21 [==============================] - 0s 4ms/step - loss: 3.7872 - accuracy: 0.0258 - val_loss: 3.7687 - val_accuracy: 0.0177
Epoch 25/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7785 - accuracy: 0.0243 - val_loss: 3.8283 - val_accuracy: 0.0213
Epoch 26/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7762 - accuracy: 0.0410 - val_loss: 3.7586 - val_accuracy: 0.0177
Epoch 27/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7826 - accuracy: 0.0334 - val_loss: 3.8165 - val_accuracy: 0.0213
Epoch 28/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7501 - accuracy: 0.0289 - val_loss: 3.8179 - val_accuracy: 0.0177
Epoch 29/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7713 - accuracy: 0.0319 - val_loss: 3.7451 - val_accuracy: 0.0177
Epoch 30/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7681 - accuracy: 0.0410 - val_loss: 3.7596 - val_accuracy: 0.0177
Epoch 31/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7470 - accuracy: 0.0350 - val_loss: 3.7624 - val_accuracy: 0.0213
Epoch 32/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7323 - accuracy: 0.0395 - val_loss: 3.8113 - val_accuracy: 0.0213
Epoch 33/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7594 - accuracy: 0.0426 - val_loss: 3.7654 - val_accuracy: 0.0213
Epoch 34/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7179 - accuracy: 0.0350 - val_loss: 3.8115 - val_accuracy: 0.0213
Epoch 35/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7440 - accuracy: 0.0334 - val_loss: 3.7497 - val_accuracy: 0.0213
Epoch 36/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7035 - accuracy: 0.0258 - val_loss: 3.7566 - val_accuracy: 0.0213
Epoch 37/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7108 - accuracy: 0.0319 - val_loss: 3.8110 - val_accuracy: 0.0213
Epoch 38/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7032 - accuracy: 0.0289 - val_loss: 3.6679 - val_accuracy: 0.0177
Epoch 39/100
21/21 [==============================] - 0s 3ms/step - loss: 3.6573 - accuracy: 0.0228 - val_loss: 3.7131 - val_accuracy: 0.0177
Epoch 40/100
21/21 [==============================] - 0s 3ms/step - loss: 3.6707 - accuracy: 0.0274 - val_loss: 3.7150 - val_accuracy: 0.0177
Epoch 41/100
21/21 [==============================] - 0s 4ms/step - loss: 3.6636 - accuracy: 0.0258 - val_loss: 3.8201 - val_accuracy: 0.0213
Epoch 42/100
21/21 [==============================] - 0s 4ms/step - loss: 3.7550 - accuracy: 0.0441 - val_loss: 3.6304 - val_accuracy: 0.0177
Epoch 43/100
21/21 [==============================] - 0s 4ms/step - loss: 3.6356 - accuracy: 0.0243 - val_loss: 3.6168 - val_accuracy: 0.0177
Epoch 44/100
21/21 [==============================] - 0s 3ms/step - loss: 3.6505 - accuracy: 0.0532 - val_loss: 3.6092 - val_accuracy: 0.0248
Epoch 45/100
21/21 [==============================] - 0s 3ms/step - loss: 3.6004 - accuracy: 0.0517 - val_loss: 3.6050 - val_accuracy: 0.0248
Epoch 46/100
21/21 [==============================] - 0s 3ms/step - loss: 3.6023 - accuracy: 0.0456 - val_loss: 3.5991 - val_accuracy: 0.0248
Epoch 47/100
21/21 [==============================] - 0s 3ms/step - loss: 3.6858 - accuracy: 0.0441 - val_loss: 3.8104 - val_accuracy: 0.0248
Epoch 48/100
21/21 [==============================] - 0s 4ms/step - loss: 3.6234 - accuracy: 0.0517 - val_loss: 3.5895 - val_accuracy: 0.0248
Epoch 49/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5888 - accuracy: 0.0486 - val_loss: 3.5874 - val_accuracy: 0.0248
Epoch 50/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5675 - accuracy: 0.0486 - val_loss: 3.5750 - val_accuracy: 0.0248
Epoch 51/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5661 - accuracy: 0.0547 - val_loss: 3.6012 - val_accuracy: 0.0248
Epoch 52/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5781 - accuracy: 0.0562 - val_loss: 3.5698 - val_accuracy: 0.0248
Epoch 53/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5633 - accuracy: 0.0517 - val_loss: 3.5678 - val_accuracy: 0.0248
Epoch 54/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5579 - accuracy: 0.0517 - val_loss: 3.5575 - val_accuracy: 0.0248
Epoch 55/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5433 - accuracy: 0.0502 - val_loss: 3.5671 - val_accuracy: 0.0248
Epoch 56/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5477 - accuracy: 0.0426 - val_loss: 3.5610 - val_accuracy: 0.0248
Epoch 57/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5453 - accuracy: 0.0532 - val_loss: 3.5900 - val_accuracy: 0.0248
Epoch 58/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5359 - accuracy: 0.0502 - val_loss: 3.5496 - val_accuracy: 0.0390
Epoch 59/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5560 - accuracy: 0.0578 - val_loss: 3.5548 - val_accuracy: 0.0248
Epoch 60/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5289 - accuracy: 0.0502 - val_loss: 3.5567 - val_accuracy: 0.0248
Epoch 61/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5300 - accuracy: 0.0502 - val_loss: 3.5404 - val_accuracy: 0.0248
Epoch 62/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5253 - accuracy: 0.0578 - val_loss: 3.5377 - val_accuracy: 0.0248
Epoch 63/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5191 - accuracy: 0.0486 - val_loss: 3.5348 - val_accuracy: 0.0248
Epoch 64/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5327 - accuracy: 0.0532 - val_loss: 3.5337 - val_accuracy: 0.0142
Epoch 65/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5121 - accuracy: 0.0547 - val_loss: 3.5407 - val_accuracy: 0.0248
Epoch 66/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5211 - accuracy: 0.0486 - val_loss: 3.5370 - val_accuracy: 0.0248
Epoch 67/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5087 - accuracy: 0.0547 - val_loss: 3.5410 - val_accuracy: 0.0461
Epoch 68/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4652 - accuracy: 0.0669 - val_loss: 3.6191 - val_accuracy: 0.0532
Epoch 69/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4913 - accuracy: 0.0729 - val_loss: 3.5274 - val_accuracy: 0.0426
Epoch 70/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4713 - accuracy: 0.0623 - val_loss: 3.5469 - val_accuracy: 0.0248
Epoch 71/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5066 - accuracy: 0.0441 - val_loss: 3.5392 - val_accuracy: 0.0248
Epoch 72/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5066 - accuracy: 0.0532 - val_loss: 3.5302 - val_accuracy: 0.0248
Epoch 73/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4997 - accuracy: 0.0517 - val_loss: 3.5379 - val_accuracy: 0.0248
Epoch 74/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4970 - accuracy: 0.0471 - val_loss: 3.5351 - val_accuracy: 0.0248
Epoch 75/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5022 - accuracy: 0.0502 - val_loss: 3.5280 - val_accuracy: 0.0248
Epoch 76/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4876 - accuracy: 0.0456 - val_loss: 3.4635 - val_accuracy: 0.0355
Epoch 77/100
21/21 [==============================] - 0s 3ms/step - loss: 3.3869 - accuracy: 0.0684 - val_loss: 3.4348 - val_accuracy: 0.0319
Epoch 78/100
21/21 [==============================] - 0s 3ms/step - loss: 3.3410 - accuracy: 0.0927 - val_loss: 3.4583 - val_accuracy: 0.0390
Epoch 79/100
21/21 [==============================] - 0s 3ms/step - loss: 3.3423 - accuracy: 0.0729 - val_loss: 3.3591 - val_accuracy: 0.0390
Epoch 80/100
21/21 [==============================] - 0s 3ms/step - loss: 3.2987 - accuracy: 0.0745 - val_loss: 3.4295 - val_accuracy: 0.0426
Epoch 81/100
21/21 [==============================] - 0s 3ms/step - loss: 3.2729 - accuracy: 0.0775 - val_loss: 3.3685 - val_accuracy: 0.0426
Epoch 82/100
21/21 [==============================] - 0s 3ms/step - loss: 3.2259 - accuracy: 0.0805 - val_loss: 3.3374 - val_accuracy: 0.0496
Epoch 83/100
21/21 [==============================] - 0s 3ms/step - loss: 3.2412 - accuracy: 0.0745 - val_loss: 3.3074 - val_accuracy: 0.0319
Epoch 84/100
21/21 [==============================] - 0s 3ms/step - loss: 3.2263 - accuracy: 0.0745 - val_loss: 3.3043 - val_accuracy: 0.0426
Epoch 85/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1720 - accuracy: 0.0836 - val_loss: 3.2140 - val_accuracy: 0.0319
Epoch 86/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1747 - accuracy: 0.0851 - val_loss: 3.2157 - val_accuracy: 0.0390
Epoch 87/100
21/21 [==============================] - 0s 4ms/step - loss: 3.1288 - accuracy: 0.0805 - val_loss: 3.2095 - val_accuracy: 0.0496
Epoch 88/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1460 - accuracy: 0.0836 - val_loss: 3.1958 - val_accuracy: 0.0426
Epoch 89/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1673 - accuracy: 0.0942 - val_loss: 3.2467 - val_accuracy: 0.0496
Epoch 90/100
21/21 [==============================] - 0s 3ms/step - loss: 3.2416 - accuracy: 0.0942 - val_loss: 3.1322 - val_accuracy: 0.0851
Epoch 91/100
21/21 [==============================] - 0s 3ms/step - loss: 3.0884 - accuracy: 0.0760 - val_loss: 3.1839 - val_accuracy: 0.1064
Epoch 92/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1396 - accuracy: 0.0866 - val_loss: 3.2761 - val_accuracy: 0.0461
Epoch 93/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1014 - accuracy: 0.0866 - val_loss: 3.1126 - val_accuracy: 0.0922
Epoch 94/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1292 - accuracy: 0.0775 - val_loss: 3.5850 - val_accuracy: 0.0355
Epoch 95/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1564 - accuracy: 0.0957 - val_loss: 3.1161 - val_accuracy: 0.0390
Epoch 96/100
21/21 [==============================] - 0s 3ms/step - loss: 3.0917 - accuracy: 0.0912 - val_loss: 3.0980 - val_accuracy: 0.0567
Epoch 97/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1061 - accuracy: 0.0760 - val_loss: 3.0975 - val_accuracy: 0.0390
Epoch 98/100
21/21 [==============================] - 0s 3ms/step - loss: 3.0769 - accuracy: 0.0912 - val_loss: 3.2535 - val_accuracy: 0.0496
Epoch 99/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1417 - accuracy: 0.0775 - val_loss: 3.2624 - val_accuracy: 0.0390
Epoch 100/100
21/21 [==============================] - 0s 3ms/step - loss: 3.0823 - accuracy: 0.0805 - val_loss: 3.1181 - val_accuracy: 0.1277
In [58]:
y_pred_2 = model_2.predict(X_test_tf)

9/9 [==============================] - 0s 2ms/step
In [59]:
model_2.evaluate(X_test_tf, y_test_tf)

9/9 [==============================] - 0s 1000us/step - loss: 3.1181 - accuracy: 0.1277
Out[59]:
[3.1180949211120605, 0.12765957415103912]
In [60]:
df_history_2 = pd.DataFrame(history_2.history)
In [61]:
fig = px.line(df_history_2, y=['loss', 'val_loss'], labels=['Loss', 'Validation Loss'])
fig.show()

Model 3: With SGD and reduced learing rate¶

In [62]:
model_3 = tf.keras.Sequential([
    layers.Input(shape=(X_train.shape[1],)),
    layers.Dense(256, activation="relu"),
    layers.Dense(47, activation="softmax")
])

model_3.compile(loss="sparse_categorical_crossentropy", optimizer=optimizers.SGD(learning_rate=0.001), metrics=["accuracy"])
In [63]:
history_3 = model_3.fit(X_train_tf, y_train_tf, epochs=100, validation_data=(X_test, y_test))

Epoch 1/100
21/21 [==============================] - 0s 9ms/step - loss: 14.4892 - accuracy: 0.0213 - val_loss: 8.4357 - val_accuracy: 0.0567
Epoch 2/100
21/21 [==============================] - 0s 4ms/step - loss: 6.5982 - accuracy: 0.0426 - val_loss: 6.2855 - val_accuracy: 0.0213
Epoch 3/100
21/21 [==============================] - 0s 3ms/step - loss: 4.8942 - accuracy: 0.0532 - val_loss: 5.5672 - val_accuracy: 0.0284
Epoch 4/100
21/21 [==============================] - 0s 4ms/step - loss: 4.9371 - accuracy: 0.0684 - val_loss: 7.3668 - val_accuracy: 0.0532
Epoch 5/100
21/21 [==============================] - 0s 5ms/step - loss: 5.0061 - accuracy: 0.0745 - val_loss: 4.7228 - val_accuracy: 0.0780
Epoch 6/100
21/21 [==============================] - 0s 4ms/step - loss: 4.2409 - accuracy: 0.0836 - val_loss: 4.6796 - val_accuracy: 0.0390
Epoch 7/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8777 - accuracy: 0.0684 - val_loss: 4.4357 - val_accuracy: 0.0461
Epoch 8/100
21/21 [==============================] - 0s 4ms/step - loss: 3.6241 - accuracy: 0.1216 - val_loss: 4.6796 - val_accuracy: 0.0851
Epoch 9/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7667 - accuracy: 0.1246 - val_loss: 4.5609 - val_accuracy: 0.0319
Epoch 10/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4184 - accuracy: 0.1459 - val_loss: 3.4874 - val_accuracy: 0.1454
Epoch 11/100
21/21 [==============================] - 0s 3ms/step - loss: 3.3562 - accuracy: 0.1611 - val_loss: 3.5428 - val_accuracy: 0.1028
Epoch 12/100
21/21 [==============================] - 0s 3ms/step - loss: 3.6389 - accuracy: 0.1125 - val_loss: 3.6217 - val_accuracy: 0.1560
Epoch 13/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1542 - accuracy: 0.1869 - val_loss: 3.4869 - val_accuracy: 0.1277
Epoch 14/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4395 - accuracy: 0.1565 - val_loss: 3.2579 - val_accuracy: 0.1773
Epoch 15/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1312 - accuracy: 0.1611 - val_loss: 4.1657 - val_accuracy: 0.0674
Epoch 16/100
21/21 [==============================] - 0s 3ms/step - loss: 3.2345 - accuracy: 0.1900 - val_loss: 3.4449 - val_accuracy: 0.0922
Epoch 17/100
21/21 [==============================] - 0s 3ms/step - loss: 3.0898 - accuracy: 0.1748 - val_loss: 3.0265 - val_accuracy: 0.1879
Epoch 18/100
21/21 [==============================] - 0s 3ms/step - loss: 2.8511 - accuracy: 0.2356 - val_loss: 3.1371 - val_accuracy: 0.1489
Epoch 19/100
21/21 [==============================] - 0s 3ms/step - loss: 2.8250 - accuracy: 0.2097 - val_loss: 3.2919 - val_accuracy: 0.1064
Epoch 20/100
21/21 [==============================] - 0s 3ms/step - loss: 2.9119 - accuracy: 0.1930 - val_loss: 3.1271 - val_accuracy: 0.1525
Epoch 21/100
21/21 [==============================] - 0s 5ms/step - loss: 2.8047 - accuracy: 0.1945 - val_loss: 2.8109 - val_accuracy: 0.2376
Epoch 22/100
21/21 [==============================] - 0s 3ms/step - loss: 2.7372 - accuracy: 0.2629 - val_loss: 3.1432 - val_accuracy: 0.2021
Epoch 23/100
21/21 [==============================] - 0s 3ms/step - loss: 2.8168 - accuracy: 0.2067 - val_loss: 3.3895 - val_accuracy: 0.1241
Epoch 24/100
21/21 [==============================] - 0s 4ms/step - loss: 2.7250 - accuracy: 0.2796 - val_loss: 3.1474 - val_accuracy: 0.1667
Epoch 25/100
21/21 [==============================] - 0s 3ms/step - loss: 2.7716 - accuracy: 0.2264 - val_loss: 2.7814 - val_accuracy: 0.1241
Epoch 26/100
21/21 [==============================] - 0s 3ms/step - loss: 2.7485 - accuracy: 0.2416 - val_loss: 2.7163 - val_accuracy: 0.3050
Epoch 27/100
21/21 [==============================] - 0s 3ms/step - loss: 2.6468 - accuracy: 0.2492 - val_loss: 3.0344 - val_accuracy: 0.1667
Epoch 28/100
21/21 [==============================] - 0s 3ms/step - loss: 2.6338 - accuracy: 0.2629 - val_loss: 2.7243 - val_accuracy: 0.1383
Epoch 29/100
21/21 [==============================] - 0s 3ms/step - loss: 2.5593 - accuracy: 0.2599 - val_loss: 2.7079 - val_accuracy: 0.2447
Epoch 30/100
21/21 [==============================] - 0s 3ms/step - loss: 2.6200 - accuracy: 0.2584 - val_loss: 2.7093 - val_accuracy: 0.1348
Epoch 31/100
21/21 [==============================] - 0s 3ms/step - loss: 2.6084 - accuracy: 0.2553 - val_loss: 2.5217 - val_accuracy: 0.2340
Epoch 32/100
21/21 [==============================] - 0s 3ms/step - loss: 2.6224 - accuracy: 0.2432 - val_loss: 2.7639 - val_accuracy: 0.1879
Epoch 33/100
21/21 [==============================] - 0s 3ms/step - loss: 2.5631 - accuracy: 0.2872 - val_loss: 2.6634 - val_accuracy: 0.1667
Epoch 34/100
21/21 [==============================] - 0s 3ms/step - loss: 2.5208 - accuracy: 0.2964 - val_loss: 2.8945 - val_accuracy: 0.2021
Epoch 35/100
21/21 [==============================] - 0s 3ms/step - loss: 2.4612 - accuracy: 0.2933 - val_loss: 2.5743 - val_accuracy: 0.2199
Epoch 36/100
21/21 [==============================] - 0s 3ms/step - loss: 2.4215 - accuracy: 0.2766 - val_loss: 2.6795 - val_accuracy: 0.2234
Epoch 37/100
21/21 [==============================] - 0s 3ms/step - loss: 2.5198 - accuracy: 0.2766 - val_loss: 2.7578 - val_accuracy: 0.2021
Epoch 38/100
21/21 [==============================] - 0s 3ms/step - loss: 2.4607 - accuracy: 0.3435 - val_loss: 2.7972 - val_accuracy: 0.1667
Epoch 39/100
21/21 [==============================] - 0s 3ms/step - loss: 2.4768 - accuracy: 0.2964 - val_loss: 2.6415 - val_accuracy: 0.1986
Epoch 40/100
21/21 [==============================] - 0s 3ms/step - loss: 2.4624 - accuracy: 0.2812 - val_loss: 2.5825 - val_accuracy: 0.1986
Epoch 41/100
21/21 [==============================] - 0s 4ms/step - loss: 2.3713 - accuracy: 0.3541 - val_loss: 2.5483 - val_accuracy: 0.2518
Epoch 42/100
21/21 [==============================] - 0s 3ms/step - loss: 2.3607 - accuracy: 0.3100 - val_loss: 2.5171 - val_accuracy: 0.2092
Epoch 43/100
21/21 [==============================] - 0s 3ms/step - loss: 2.3741 - accuracy: 0.3207 - val_loss: 3.2491 - val_accuracy: 0.1596
Epoch 44/100
21/21 [==============================] - 0s 3ms/step - loss: 2.4644 - accuracy: 0.2857 - val_loss: 2.3219 - val_accuracy: 0.3404
Epoch 45/100
21/21 [==============================] - 0s 3ms/step - loss: 2.2972 - accuracy: 0.3435 - val_loss: 2.4395 - val_accuracy: 0.3191
Epoch 46/100
21/21 [==============================] - 0s 3ms/step - loss: 2.4729 - accuracy: 0.2766 - val_loss: 2.9337 - val_accuracy: 0.1844
Epoch 47/100
21/21 [==============================] - 0s 3ms/step - loss: 2.2877 - accuracy: 0.3419 - val_loss: 2.2959 - val_accuracy: 0.3688
Epoch 48/100
21/21 [==============================] - 0s 3ms/step - loss: 2.2810 - accuracy: 0.3419 - val_loss: 3.1424 - val_accuracy: 0.2801
Epoch 49/100
21/21 [==============================] - 0s 3ms/step - loss: 2.3353 - accuracy: 0.3495 - val_loss: 2.5023 - val_accuracy: 0.1844
Epoch 50/100
21/21 [==============================] - 0s 3ms/step - loss: 2.2539 - accuracy: 0.3647 - val_loss: 2.5889 - val_accuracy: 0.1418
Epoch 51/100
21/21 [==============================] - 0s 2ms/step - loss: 2.2849 - accuracy: 0.3252 - val_loss: 2.3828 - val_accuracy: 0.2801
Epoch 52/100
21/21 [==============================] - 0s 2ms/step - loss: 2.2470 - accuracy: 0.3374 - val_loss: 2.6475 - val_accuracy: 0.1915
Epoch 53/100
21/21 [==============================] - 0s 3ms/step - loss: 2.2288 - accuracy: 0.3602 - val_loss: 2.2823 - val_accuracy: 0.3156
Epoch 54/100
21/21 [==============================] - 0s 3ms/step - loss: 2.1836 - accuracy: 0.3587 - val_loss: 2.2737 - val_accuracy: 0.3333
Epoch 55/100
21/21 [==============================] - 0s 3ms/step - loss: 2.2491 - accuracy: 0.3663 - val_loss: 2.5052 - val_accuracy: 0.2979
Epoch 56/100
21/21 [==============================] - 0s 3ms/step - loss: 2.1760 - accuracy: 0.3602 - val_loss: 2.4490 - val_accuracy: 0.2163
Epoch 57/100
21/21 [==============================] - 0s 4ms/step - loss: 2.1378 - accuracy: 0.3891 - val_loss: 2.3707 - val_accuracy: 0.3227
Epoch 58/100
21/21 [==============================] - 0s 4ms/step - loss: 2.1169 - accuracy: 0.3693 - val_loss: 2.2789 - val_accuracy: 0.3582
Epoch 59/100
21/21 [==============================] - 0s 3ms/step - loss: 2.1599 - accuracy: 0.3663 - val_loss: 2.2884 - val_accuracy: 0.3582
Epoch 60/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0927 - accuracy: 0.3936 - val_loss: 2.7046 - val_accuracy: 0.2199
Epoch 61/100
21/21 [==============================] - 0s 3ms/step - loss: 2.1461 - accuracy: 0.3815 - val_loss: 2.4644 - val_accuracy: 0.2872
Epoch 62/100
21/21 [==============================] - 0s 3ms/step - loss: 2.1025 - accuracy: 0.3815 - val_loss: 2.8568 - val_accuracy: 0.2092
Epoch 63/100
21/21 [==============================] - 0s 3ms/step - loss: 2.1007 - accuracy: 0.3708 - val_loss: 2.2001 - val_accuracy: 0.4113
Epoch 64/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0423 - accuracy: 0.3906 - val_loss: 2.2326 - val_accuracy: 0.3440
Epoch 65/100
21/21 [==============================] - 0s 3ms/step - loss: 2.1045 - accuracy: 0.3799 - val_loss: 2.3143 - val_accuracy: 0.2979
Epoch 66/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0882 - accuracy: 0.3708 - val_loss: 2.3161 - val_accuracy: 0.3050
Epoch 67/100
21/21 [==============================] - 0s 3ms/step - loss: 2.1004 - accuracy: 0.3875 - val_loss: 2.2413 - val_accuracy: 0.3865
Epoch 68/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0648 - accuracy: 0.4134 - val_loss: 2.2944 - val_accuracy: 0.2979
Epoch 69/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0142 - accuracy: 0.4255 - val_loss: 2.2155 - val_accuracy: 0.3156
Epoch 70/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0229 - accuracy: 0.4179 - val_loss: 2.1015 - val_accuracy: 0.4078
Epoch 71/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0482 - accuracy: 0.4210 - val_loss: 2.1063 - val_accuracy: 0.3475
Epoch 72/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0233 - accuracy: 0.4179 - val_loss: 2.4181 - val_accuracy: 0.2270
Epoch 73/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0023 - accuracy: 0.4331 - val_loss: 2.1582 - val_accuracy: 0.3688
Epoch 74/100
21/21 [==============================] - 0s 2ms/step - loss: 1.9476 - accuracy: 0.4711 - val_loss: 2.3384 - val_accuracy: 0.3652
Epoch 75/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9689 - accuracy: 0.4377 - val_loss: 2.4184 - val_accuracy: 0.3298
Epoch 76/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0148 - accuracy: 0.4210 - val_loss: 2.1212 - val_accuracy: 0.2943
Epoch 77/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9875 - accuracy: 0.4195 - val_loss: 2.1274 - val_accuracy: 0.3050
Epoch 78/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9610 - accuracy: 0.4407 - val_loss: 2.3009 - val_accuracy: 0.3227
Epoch 79/100
21/21 [==============================] - 0s 4ms/step - loss: 1.9883 - accuracy: 0.4179 - val_loss: 2.2773 - val_accuracy: 0.3511
Epoch 80/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9420 - accuracy: 0.4195 - val_loss: 2.0318 - val_accuracy: 0.3830
Epoch 81/100
21/21 [==============================] - 0s 4ms/step - loss: 1.9735 - accuracy: 0.4164 - val_loss: 2.1844 - val_accuracy: 0.2943
Epoch 82/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9507 - accuracy: 0.4362 - val_loss: 2.1373 - val_accuracy: 0.4220
Epoch 83/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9139 - accuracy: 0.4255 - val_loss: 2.2248 - val_accuracy: 0.3121
Epoch 84/100
21/21 [==============================] - 0s 2ms/step - loss: 1.9648 - accuracy: 0.4347 - val_loss: 2.1209 - val_accuracy: 0.3972
Epoch 85/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9527 - accuracy: 0.4164 - val_loss: 2.0651 - val_accuracy: 0.3404
Epoch 86/100
21/21 [==============================] - 0s 4ms/step - loss: 1.8993 - accuracy: 0.4514 - val_loss: 2.1670 - val_accuracy: 0.3475
Epoch 87/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8762 - accuracy: 0.4711 - val_loss: 2.0003 - val_accuracy: 0.4433
Epoch 88/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8697 - accuracy: 0.4650 - val_loss: 1.9363 - val_accuracy: 0.4539
Epoch 89/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9141 - accuracy: 0.4301 - val_loss: 2.4761 - val_accuracy: 0.2695
Epoch 90/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9037 - accuracy: 0.4438 - val_loss: 2.0287 - val_accuracy: 0.3546
Epoch 91/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8643 - accuracy: 0.4544 - val_loss: 1.9852 - val_accuracy: 0.3262
Epoch 92/100
21/21 [==============================] - 0s 2ms/step - loss: 1.8288 - accuracy: 0.4802 - val_loss: 1.8983 - val_accuracy: 0.4504
Epoch 93/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8636 - accuracy: 0.4559 - val_loss: 1.9676 - val_accuracy: 0.3440
Epoch 94/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8469 - accuracy: 0.4666 - val_loss: 2.2154 - val_accuracy: 0.3369
Epoch 95/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8647 - accuracy: 0.4498 - val_loss: 2.2801 - val_accuracy: 0.3050
Epoch 96/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8551 - accuracy: 0.4483 - val_loss: 1.9022 - val_accuracy: 0.4468
Epoch 97/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8276 - accuracy: 0.4635 - val_loss: 2.1067 - val_accuracy: 0.4539
Epoch 98/100
21/21 [==============================] - 0s 3ms/step - loss: 1.7975 - accuracy: 0.4818 - val_loss: 1.9873 - val_accuracy: 0.3440
Epoch 99/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8014 - accuracy: 0.4954 - val_loss: 2.0932 - val_accuracy: 0.3617
Epoch 100/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8349 - accuracy: 0.4650 - val_loss: 2.0703 - val_accuracy: 0.3298
In [64]:
y_pred_3 = model_3.predict(X_test_tf)

9/9 [==============================] - 0s 1ms/step
In [65]:
model_3.evaluate(X_test_tf, y_test_tf)

9/9 [==============================] - 0s 2ms/step - loss: 2.0703 - accuracy: 0.3298
Out[65]:
[2.0702500343322754, 0.3297872245311737]
In [66]:
df_history_3 = pd.DataFrame(history_3.history)
In [67]:
fig = px.line(df_history_3, y=['loss', 'val_loss'], labels=['Loss', 'Validation Loss'])
fig.show()

Model 4: SGD with 0.005 learning rate¶

In [68]:
model_4 = tf.keras.Sequential([
    layers.Input(shape=(X_train.shape[1],)),
    layers.Dense(256, activation="relu"),
    layers.Dense(47, activation="softmax")
])

model_4.compile(loss="sparse_categorical_crossentropy", optimizer=optimizers.SGD(learning_rate=0.005), metrics=["accuracy"])
In [69]:
history_4 = model_4.fit(X_train_tf, y_train_tf, epochs=100, validation_data=(X_test, y_test))

Epoch 1/100
21/21 [==============================] - 0s 8ms/step - loss: 19.1908 - accuracy: 0.0289 - val_loss: 3.7442 - val_accuracy: 0.1241
Epoch 2/100
21/21 [==============================] - 0s 4ms/step - loss: 3.7318 - accuracy: 0.0760 - val_loss: 3.7503 - val_accuracy: 0.0887
Epoch 3/100
21/21 [==============================] - 0s 3ms/step - loss: 3.7069 - accuracy: 0.0593 - val_loss: 3.6895 - val_accuracy: 0.1028
Epoch 4/100
21/21 [==============================] - 0s 3ms/step - loss: 3.6832 - accuracy: 0.0912 - val_loss: 3.6896 - val_accuracy: 0.0709
Epoch 5/100
21/21 [==============================] - 0s 4ms/step - loss: 3.6758 - accuracy: 0.0881 - val_loss: 3.7536 - val_accuracy: 0.1028
Epoch 6/100
21/21 [==============================] - 0s 4ms/step - loss: 3.6305 - accuracy: 0.0973 - val_loss: 3.7053 - val_accuracy: 0.0922
Epoch 7/100
21/21 [==============================] - 0s 4ms/step - loss: 3.5982 - accuracy: 0.1125 - val_loss: 3.6539 - val_accuracy: 0.0851
Epoch 8/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5693 - accuracy: 0.1277 - val_loss: 3.5997 - val_accuracy: 0.1099
Epoch 9/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5082 - accuracy: 0.1353 - val_loss: 3.6330 - val_accuracy: 0.0922
Epoch 10/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4939 - accuracy: 0.1398 - val_loss: 3.5490 - val_accuracy: 0.0922
Epoch 11/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4220 - accuracy: 0.1398 - val_loss: 3.6058 - val_accuracy: 0.0851
Epoch 12/100
21/21 [==============================] - 0s 3ms/step - loss: 3.4100 - accuracy: 0.1246 - val_loss: 3.5620 - val_accuracy: 0.1348
Epoch 13/100
21/21 [==============================] - 0s 3ms/step - loss: 3.3485 - accuracy: 0.1261 - val_loss: 3.4304 - val_accuracy: 0.0709
Epoch 14/100
21/21 [==============================] - 0s 3ms/step - loss: 3.3169 - accuracy: 0.1261 - val_loss: 3.4282 - val_accuracy: 0.1099
Epoch 15/100
21/21 [==============================] - 0s 3ms/step - loss: 3.3240 - accuracy: 0.1216 - val_loss: 3.4061 - val_accuracy: 0.0674
Epoch 16/100
21/21 [==============================] - 0s 3ms/step - loss: 3.2348 - accuracy: 0.1565 - val_loss: 3.4993 - val_accuracy: 0.1099
Epoch 17/100
21/21 [==============================] - 0s 3ms/step - loss: 3.1987 - accuracy: 0.1444 - val_loss: 3.3551 - val_accuracy: 0.1028
Epoch 18/100
21/21 [==============================] - 0s 5ms/step - loss: 3.1453 - accuracy: 0.1596 - val_loss: 3.4233 - val_accuracy: 0.0851
Epoch 19/100
21/21 [==============================] - 0s 4ms/step - loss: 3.0770 - accuracy: 0.1565 - val_loss: 3.3256 - val_accuracy: 0.0887
Epoch 20/100
21/21 [==============================] - 0s 5ms/step - loss: 3.0439 - accuracy: 0.1657 - val_loss: 3.0617 - val_accuracy: 0.1915
Epoch 21/100
21/21 [==============================] - 0s 5ms/step - loss: 3.0051 - accuracy: 0.1839 - val_loss: 3.1023 - val_accuracy: 0.1312
Epoch 22/100
21/21 [==============================] - 0s 5ms/step - loss: 2.9373 - accuracy: 0.1854 - val_loss: 3.2413 - val_accuracy: 0.1631
Epoch 23/100
21/21 [==============================] - 0s 4ms/step - loss: 2.9136 - accuracy: 0.1778 - val_loss: 3.0983 - val_accuracy: 0.1383
Epoch 24/100
21/21 [==============================] - 0s 3ms/step - loss: 2.8203 - accuracy: 0.1854 - val_loss: 2.9578 - val_accuracy: 0.1667
Epoch 25/100
21/21 [==============================] - 0s 3ms/step - loss: 2.7999 - accuracy: 0.2112 - val_loss: 2.9393 - val_accuracy: 0.1738
Epoch 26/100
21/21 [==============================] - 0s 3ms/step - loss: 2.7427 - accuracy: 0.2082 - val_loss: 2.9343 - val_accuracy: 0.1702
Epoch 27/100
21/21 [==============================] - 0s 3ms/step - loss: 2.7112 - accuracy: 0.2143 - val_loss: 2.7592 - val_accuracy: 0.2057
Epoch 28/100
21/21 [==============================] - 0s 3ms/step - loss: 2.6348 - accuracy: 0.2264 - val_loss: 2.7777 - val_accuracy: 0.2128
Epoch 29/100
21/21 [==============================] - 0s 3ms/step - loss: 2.5542 - accuracy: 0.2538 - val_loss: 2.6700 - val_accuracy: 0.1879
Epoch 30/100
21/21 [==============================] - 0s 4ms/step - loss: 2.5250 - accuracy: 0.2644 - val_loss: 2.6458 - val_accuracy: 0.1667
Epoch 31/100
21/21 [==============================] - 0s 3ms/step - loss: 2.4787 - accuracy: 0.2948 - val_loss: 2.6180 - val_accuracy: 0.1986
Epoch 32/100
21/21 [==============================] - 0s 3ms/step - loss: 2.4598 - accuracy: 0.2447 - val_loss: 2.4219 - val_accuracy: 0.2624
Epoch 33/100
21/21 [==============================] - 0s 3ms/step - loss: 2.3412 - accuracy: 0.2964 - val_loss: 2.5017 - val_accuracy: 0.2163
Epoch 34/100
21/21 [==============================] - 0s 3ms/step - loss: 2.3286 - accuracy: 0.3267 - val_loss: 2.5064 - val_accuracy: 0.1844
Epoch 35/100
21/21 [==============================] - 0s 3ms/step - loss: 2.3252 - accuracy: 0.3085 - val_loss: 2.4128 - val_accuracy: 0.1950
Epoch 36/100
21/21 [==============================] - 0s 3ms/step - loss: 2.2664 - accuracy: 0.3191 - val_loss: 2.3075 - val_accuracy: 0.2872
Epoch 37/100
21/21 [==============================] - 0s 3ms/step - loss: 2.2031 - accuracy: 0.3450 - val_loss: 2.4057 - val_accuracy: 0.1986
Epoch 38/100
21/21 [==============================] - 0s 4ms/step - loss: 2.1000 - accuracy: 0.3967 - val_loss: 2.3282 - val_accuracy: 0.3262
Epoch 39/100
21/21 [==============================] - 0s 4ms/step - loss: 2.1212 - accuracy: 0.3906 - val_loss: 2.2808 - val_accuracy: 0.2730
Epoch 40/100
21/21 [==============================] - 0s 4ms/step - loss: 2.0923 - accuracy: 0.4027 - val_loss: 2.3329 - val_accuracy: 0.2128
Epoch 41/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0799 - accuracy: 0.3845 - val_loss: 2.2036 - val_accuracy: 0.2695
Epoch 42/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0314 - accuracy: 0.3906 - val_loss: 2.0455 - val_accuracy: 0.3121
Epoch 43/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9813 - accuracy: 0.4088 - val_loss: 2.0671 - val_accuracy: 0.4078
Epoch 44/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9865 - accuracy: 0.4271 - val_loss: 2.3357 - val_accuracy: 0.2908
Epoch 45/100
21/21 [==============================] - 0s 3ms/step - loss: 2.0552 - accuracy: 0.3647 - val_loss: 2.0756 - val_accuracy: 0.3050
Epoch 46/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9413 - accuracy: 0.4225 - val_loss: 2.0029 - val_accuracy: 0.3936
Epoch 47/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8788 - accuracy: 0.4483 - val_loss: 2.0009 - val_accuracy: 0.4255
Epoch 48/100
21/21 [==============================] - 0s 2ms/step - loss: 1.8806 - accuracy: 0.4590 - val_loss: 1.9303 - val_accuracy: 0.4149
Epoch 49/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9137 - accuracy: 0.4271 - val_loss: 2.0156 - val_accuracy: 0.3617
Epoch 50/100
21/21 [==============================] - 0s 3ms/step - loss: 1.9472 - accuracy: 0.3982 - val_loss: 1.8541 - val_accuracy: 0.4574
Epoch 51/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8210 - accuracy: 0.4590 - val_loss: 2.0460 - val_accuracy: 0.3440
Epoch 52/100
21/21 [==============================] - 0s 3ms/step - loss: 1.8225 - accuracy: 0.4742 - val_loss: 1.9128 - val_accuracy: 0.4787
Epoch 53/100
21/21 [==============================] - 0s 3ms/step - loss: 1.7436 - accuracy: 0.5334 - val_loss: 2.2205 - val_accuracy: 0.3369
Epoch 54/100
21/21 [==============================] - 0s 4ms/step - loss: 1.7854 - accuracy: 0.4802 - val_loss: 1.8090 - val_accuracy: 0.5319
Epoch 55/100
21/21 [==============================] - 0s 3ms/step - loss: 1.7653 - accuracy: 0.4985 - val_loss: 1.7886 - val_accuracy: 0.4716
Epoch 56/100
21/21 [==============================] - 0s 3ms/step - loss: 1.7602 - accuracy: 0.4742 - val_loss: 1.7797 - val_accuracy: 0.5177
Epoch 57/100
21/21 [==============================] - 0s 3ms/step - loss: 1.7357 - accuracy: 0.4757 - val_loss: 2.4338 - val_accuracy: 0.2624
Epoch 58/100
21/21 [==============================] - 0s 3ms/step - loss: 1.7302 - accuracy: 0.4878 - val_loss: 1.9087 - val_accuracy: 0.3298
Epoch 59/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6937 - accuracy: 0.5076 - val_loss: 1.8943 - val_accuracy: 0.4149
Epoch 60/100
21/21 [==============================] - 0s 3ms/step - loss: 1.7237 - accuracy: 0.4757 - val_loss: 1.7826 - val_accuracy: 0.4716
Epoch 61/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6383 - accuracy: 0.5334 - val_loss: 1.7604 - val_accuracy: 0.4574
Epoch 62/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6049 - accuracy: 0.5426 - val_loss: 1.6902 - val_accuracy: 0.5142
Epoch 63/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6883 - accuracy: 0.5304 - val_loss: 1.7393 - val_accuracy: 0.4965
Epoch 64/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6303 - accuracy: 0.5228 - val_loss: 1.6676 - val_accuracy: 0.4858
Epoch 65/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6203 - accuracy: 0.5258 - val_loss: 1.7056 - val_accuracy: 0.4504
Epoch 66/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6256 - accuracy: 0.5076 - val_loss: 1.7786 - val_accuracy: 0.4823
Epoch 67/100
21/21 [==============================] - 0s 3ms/step - loss: 1.5594 - accuracy: 0.5790 - val_loss: 1.7894 - val_accuracy: 0.3723
Epoch 68/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6233 - accuracy: 0.4818 - val_loss: 1.7214 - val_accuracy: 0.4433
Epoch 69/100
21/21 [==============================] - 0s 3ms/step - loss: 1.5848 - accuracy: 0.5137 - val_loss: 2.0790 - val_accuracy: 0.3121
Epoch 70/100
21/21 [==============================] - 0s 3ms/step - loss: 1.5753 - accuracy: 0.5380 - val_loss: 1.7696 - val_accuracy: 0.4255
Epoch 71/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6376 - accuracy: 0.5046 - val_loss: 1.7018 - val_accuracy: 0.4574
Epoch 72/100
21/21 [==============================] - 0s 4ms/step - loss: 1.5891 - accuracy: 0.5198 - val_loss: 1.7204 - val_accuracy: 0.4149
Epoch 73/100
21/21 [==============================] - 0s 3ms/step - loss: 1.4951 - accuracy: 0.5669 - val_loss: 1.7573 - val_accuracy: 0.4255
Epoch 74/100
21/21 [==============================] - 0s 3ms/step - loss: 1.5089 - accuracy: 0.5638 - val_loss: 1.6802 - val_accuracy: 0.4326
Epoch 75/100
21/21 [==============================] - 0s 3ms/step - loss: 1.4941 - accuracy: 0.5593 - val_loss: 1.7269 - val_accuracy: 0.5142
Epoch 76/100
21/21 [==============================] - 0s 4ms/step - loss: 1.5479 - accuracy: 0.5274 - val_loss: 1.7008 - val_accuracy: 0.4681
Epoch 77/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6475 - accuracy: 0.4544 - val_loss: 1.8921 - val_accuracy: 0.4681
Epoch 78/100
21/21 [==============================] - 0s 3ms/step - loss: 1.5451 - accuracy: 0.5091 - val_loss: 1.5821 - val_accuracy: 0.5461
Epoch 79/100
21/21 [==============================] - 0s 4ms/step - loss: 1.4566 - accuracy: 0.5669 - val_loss: 1.5991 - val_accuracy: 0.5106
Epoch 80/100
21/21 [==============================] - 0s 3ms/step - loss: 1.5596 - accuracy: 0.5106 - val_loss: 1.6597 - val_accuracy: 0.5035
Epoch 81/100
21/21 [==============================] - 0s 3ms/step - loss: 1.5099 - accuracy: 0.5380 - val_loss: 1.6798 - val_accuracy: 0.5177
Epoch 82/100
21/21 [==============================] - 0s 3ms/step - loss: 1.4628 - accuracy: 0.5532 - val_loss: 1.9145 - val_accuracy: 0.3333
Epoch 83/100
21/21 [==============================] - 0s 3ms/step - loss: 1.4944 - accuracy: 0.5456 - val_loss: 1.4914 - val_accuracy: 0.5603
Epoch 84/100
21/21 [==============================] - 0s 3ms/step - loss: 1.4423 - accuracy: 0.5532 - val_loss: 1.5805 - val_accuracy: 0.5709
Epoch 85/100
21/21 [==============================] - 0s 3ms/step - loss: 1.4379 - accuracy: 0.5669 - val_loss: 1.5908 - val_accuracy: 0.4574
Epoch 86/100
21/21 [==============================] - 0s 3ms/step - loss: 1.4432 - accuracy: 0.5441 - val_loss: 1.6895 - val_accuracy: 0.4752
Epoch 87/100
21/21 [==============================] - 0s 3ms/step - loss: 1.3896 - accuracy: 0.5836 - val_loss: 1.6594 - val_accuracy: 0.3936
Epoch 88/100
21/21 [==============================] - 0s 6ms/step - loss: 1.4435 - accuracy: 0.5821 - val_loss: 1.4933 - val_accuracy: 0.5390
Epoch 89/100
21/21 [==============================] - 0s 4ms/step - loss: 1.5325 - accuracy: 0.5213 - val_loss: 1.5351 - val_accuracy: 0.4716
Epoch 90/100
21/21 [==============================] - 0s 4ms/step - loss: 1.3593 - accuracy: 0.6277 - val_loss: 1.8497 - val_accuracy: 0.4255
Epoch 91/100
21/21 [==============================] - 0s 5ms/step - loss: 1.4475 - accuracy: 0.5517 - val_loss: 1.5110 - val_accuracy: 0.4645
Epoch 92/100
21/21 [==============================] - 0s 4ms/step - loss: 1.3660 - accuracy: 0.5608 - val_loss: 1.5656 - val_accuracy: 0.5426
Epoch 93/100
21/21 [==============================] - 0s 4ms/step - loss: 1.3341 - accuracy: 0.5942 - val_loss: 1.7472 - val_accuracy: 0.4184
Epoch 94/100
21/21 [==============================] - 0s 3ms/step - loss: 1.3674 - accuracy: 0.5836 - val_loss: 1.5015 - val_accuracy: 0.4823
Epoch 95/100
21/21 [==============================] - 0s 3ms/step - loss: 1.3912 - accuracy: 0.5623 - val_loss: 1.5801 - val_accuracy: 0.4858
Epoch 96/100
21/21 [==============================] - 0s 3ms/step - loss: 1.3540 - accuracy: 0.6018 - val_loss: 2.3234 - val_accuracy: 0.2199
Epoch 97/100
21/21 [==============================] - 0s 3ms/step - loss: 1.4595 - accuracy: 0.5562 - val_loss: 1.5786 - val_accuracy: 0.5496
Epoch 98/100
21/21 [==============================] - 0s 3ms/step - loss: 1.3891 - accuracy: 0.6109 - val_loss: 1.5150 - val_accuracy: 0.5709
Epoch 99/100
21/21 [==============================] - 0s 3ms/step - loss: 1.2714 - accuracy: 0.6246 - val_loss: 1.4126 - val_accuracy: 0.6028
Epoch 100/100
21/21 [==============================] - 0s 3ms/step - loss: 1.3583 - accuracy: 0.5927 - val_loss: 1.4557 - val_accuracy: 0.5000
In [70]:
y_pred_4 = model_4.predict(X_test_tf)

9/9 [==============================] - 0s 0s/step
In [71]:
model_4.evaluate(X_test_tf, y_test_tf)

9/9 [==============================] - 0s 2ms/step - loss: 1.4557 - accuracy: 0.5000
Out[71]:
[1.4557349681854248, 0.5]
In [72]:
df_history_4 = pd.DataFrame(history_4.history)
In [73]:
fig = px.line(df_history_4, y=['loss', 'val_loss'], labels=['Loss', 'Validation Loss'])
fig.show()

Model 5: With SGD and 0.05 learning rate¶

In [74]:
model_5 = tf.keras.Sequential([
    layers.Input(shape=(X_train.shape[1],)),
    layers.Dense(256, activation="relu"),
    layers.Dense(47, activation="softmax")
])

model_5.compile(loss="sparse_categorical_crossentropy", optimizer=optimizers.SGD(learning_rate=0.05), metrics=["accuracy"])
In [75]:
history_5 = model_5.fit(X_train_tf, y_train_tf, epochs=100, validation_data=(X_test, y_test))

Epoch 1/100
21/21 [==============================] - 1s 7ms/step - loss: 277.6228 - accuracy: 0.0167 - val_loss: 3.8519 - val_accuracy: 0.0035
Epoch 2/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8499 - accuracy: 0.0213 - val_loss: 3.8537 - val_accuracy: 0.0035
Epoch 3/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8492 - accuracy: 0.0137 - val_loss: 3.8553 - val_accuracy: 0.0035
Epoch 4/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8485 - accuracy: 0.0289 - val_loss: 3.8568 - val_accuracy: 0.0035
Epoch 5/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8480 - accuracy: 0.0289 - val_loss: 3.8584 - val_accuracy: 0.0035
Epoch 6/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8474 - accuracy: 0.0289 - val_loss: 3.8600 - val_accuracy: 0.0035
Epoch 7/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8469 - accuracy: 0.0289 - val_loss: 3.8616 - val_accuracy: 0.0035
Epoch 8/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8463 - accuracy: 0.0289 - val_loss: 3.8631 - val_accuracy: 0.0035
Epoch 9/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8457 - accuracy: 0.0289 - val_loss: 3.8646 - val_accuracy: 0.0035
Epoch 10/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8454 - accuracy: 0.0289 - val_loss: 3.8662 - val_accuracy: 0.0035
Epoch 11/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8448 - accuracy: 0.0289 - val_loss: 3.8676 - val_accuracy: 0.0035
Epoch 12/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8444 - accuracy: 0.0289 - val_loss: 3.8690 - val_accuracy: 0.0035
Epoch 13/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8439 - accuracy: 0.0289 - val_loss: 3.8704 - val_accuracy: 0.0035
Epoch 14/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8436 - accuracy: 0.0289 - val_loss: 3.8718 - val_accuracy: 0.0035
Epoch 15/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8432 - accuracy: 0.0289 - val_loss: 3.8732 - val_accuracy: 0.0035
Epoch 16/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8428 - accuracy: 0.0289 - val_loss: 3.8746 - val_accuracy: 0.0035
Epoch 17/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8424 - accuracy: 0.0289 - val_loss: 3.8760 - val_accuracy: 0.0035
Epoch 18/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8421 - accuracy: 0.0289 - val_loss: 3.8773 - val_accuracy: 0.0035
Epoch 19/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8417 - accuracy: 0.0289 - val_loss: 3.8785 - val_accuracy: 0.0035
Epoch 20/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8415 - accuracy: 0.0289 - val_loss: 3.8798 - val_accuracy: 0.0035
Epoch 21/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8411 - accuracy: 0.0289 - val_loss: 3.8809 - val_accuracy: 0.0035
Epoch 22/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8409 - accuracy: 0.0289 - val_loss: 3.8822 - val_accuracy: 0.0035
Epoch 23/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8406 - accuracy: 0.0289 - val_loss: 3.8834 - val_accuracy: 0.0035
Epoch 24/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8403 - accuracy: 0.0289 - val_loss: 3.8846 - val_accuracy: 0.0035
Epoch 25/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8402 - accuracy: 0.0289 - val_loss: 3.8858 - val_accuracy: 0.0035
Epoch 26/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8398 - accuracy: 0.0289 - val_loss: 3.8870 - val_accuracy: 0.0035
Epoch 27/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8395 - accuracy: 0.0289 - val_loss: 3.8881 - val_accuracy: 0.0035
Epoch 28/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8393 - accuracy: 0.0289 - val_loss: 3.8892 - val_accuracy: 0.0035
Epoch 29/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8391 - accuracy: 0.0289 - val_loss: 3.8903 - val_accuracy: 0.0035
Epoch 30/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8389 - accuracy: 0.0289 - val_loss: 3.8913 - val_accuracy: 0.0035
Epoch 31/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8388 - accuracy: 0.0289 - val_loss: 3.8923 - val_accuracy: 0.0035
Epoch 32/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8386 - accuracy: 0.0289 - val_loss: 3.8933 - val_accuracy: 0.0035
Epoch 33/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8384 - accuracy: 0.0289 - val_loss: 3.8944 - val_accuracy: 0.0035
Epoch 34/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8382 - accuracy: 0.0289 - val_loss: 3.8954 - val_accuracy: 0.0035
Epoch 35/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8380 - accuracy: 0.0289 - val_loss: 3.8964 - val_accuracy: 0.0035
Epoch 36/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8379 - accuracy: 0.0289 - val_loss: 3.8974 - val_accuracy: 0.0035
Epoch 37/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8378 - accuracy: 0.0289 - val_loss: 3.8984 - val_accuracy: 0.0035
Epoch 38/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8376 - accuracy: 0.0289 - val_loss: 3.8992 - val_accuracy: 0.0035
Epoch 39/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8375 - accuracy: 0.0289 - val_loss: 3.9001 - val_accuracy: 0.0035
Epoch 40/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8373 - accuracy: 0.0289 - val_loss: 3.9009 - val_accuracy: 0.0035
Epoch 41/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8372 - accuracy: 0.0289 - val_loss: 3.9018 - val_accuracy: 0.0035
Epoch 42/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8372 - accuracy: 0.0289 - val_loss: 3.9025 - val_accuracy: 0.0035
Epoch 43/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8370 - accuracy: 0.0289 - val_loss: 3.9034 - val_accuracy: 0.0035
Epoch 44/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8369 - accuracy: 0.0289 - val_loss: 3.9042 - val_accuracy: 0.0035
Epoch 45/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8368 - accuracy: 0.0289 - val_loss: 3.9051 - val_accuracy: 0.0035
Epoch 46/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8367 - accuracy: 0.0289 - val_loss: 3.9059 - val_accuracy: 0.0035
Epoch 47/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8366 - accuracy: 0.0289 - val_loss: 3.9066 - val_accuracy: 0.0035
Epoch 48/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8365 - accuracy: 0.0289 - val_loss: 3.9074 - val_accuracy: 0.0035
Epoch 49/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8364 - accuracy: 0.0289 - val_loss: 3.9081 - val_accuracy: 0.0035
Epoch 50/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8363 - accuracy: 0.0289 - val_loss: 3.9089 - val_accuracy: 0.0035
Epoch 51/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8362 - accuracy: 0.0289 - val_loss: 3.9096 - val_accuracy: 0.0035
Epoch 52/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8362 - accuracy: 0.0289 - val_loss: 3.9103 - val_accuracy: 0.0035
Epoch 53/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8361 - accuracy: 0.0289 - val_loss: 3.9109 - val_accuracy: 0.0035
Epoch 54/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8361 - accuracy: 0.0289 - val_loss: 3.9115 - val_accuracy: 0.0035
Epoch 55/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8360 - accuracy: 0.0289 - val_loss: 3.9120 - val_accuracy: 0.0035
Epoch 56/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8359 - accuracy: 0.0289 - val_loss: 3.9127 - val_accuracy: 0.0035
Epoch 57/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8359 - accuracy: 0.0289 - val_loss: 3.9133 - val_accuracy: 0.0035
Epoch 58/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8358 - accuracy: 0.0289 - val_loss: 3.9138 - val_accuracy: 0.0035
Epoch 59/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8358 - accuracy: 0.0289 - val_loss: 3.9145 - val_accuracy: 0.0035
Epoch 60/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8357 - accuracy: 0.0289 - val_loss: 3.9150 - val_accuracy: 0.0035
Epoch 61/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8357 - accuracy: 0.0289 - val_loss: 3.9156 - val_accuracy: 0.0035
Epoch 62/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8355 - accuracy: 0.0289 - val_loss: 3.9160 - val_accuracy: 0.0035
Epoch 63/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8356 - accuracy: 0.0289 - val_loss: 3.9166 - val_accuracy: 0.0035
Epoch 64/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8356 - accuracy: 0.0289 - val_loss: 3.9172 - val_accuracy: 0.0035
Epoch 65/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8355 - accuracy: 0.0289 - val_loss: 3.9177 - val_accuracy: 0.0035
Epoch 66/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8355 - accuracy: 0.0289 - val_loss: 3.9182 - val_accuracy: 0.0035
Epoch 67/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8354 - accuracy: 0.0289 - val_loss: 3.9188 - val_accuracy: 0.0035
Epoch 68/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8353 - accuracy: 0.0289 - val_loss: 3.9193 - val_accuracy: 0.0035
Epoch 69/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8353 - accuracy: 0.0289 - val_loss: 3.9198 - val_accuracy: 0.0035
Epoch 70/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8353 - accuracy: 0.0289 - val_loss: 3.9203 - val_accuracy: 0.0035
Epoch 71/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8352 - accuracy: 0.0289 - val_loss: 3.9208 - val_accuracy: 0.0035
Epoch 72/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8352 - accuracy: 0.0289 - val_loss: 3.9212 - val_accuracy: 0.0035
Epoch 73/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8352 - accuracy: 0.0289 - val_loss: 3.9217 - val_accuracy: 0.0035
Epoch 74/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8352 - accuracy: 0.0289 - val_loss: 3.9221 - val_accuracy: 0.0035
Epoch 75/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8351 - accuracy: 0.0289 - val_loss: 3.9225 - val_accuracy: 0.0035
Epoch 76/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8351 - accuracy: 0.0289 - val_loss: 3.9230 - val_accuracy: 0.0035
Epoch 77/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8350 - accuracy: 0.0289 - val_loss: 3.9234 - val_accuracy: 0.0035
Epoch 78/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8350 - accuracy: 0.0289 - val_loss: 3.9238 - val_accuracy: 0.0035
Epoch 79/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8350 - accuracy: 0.0289 - val_loss: 3.9241 - val_accuracy: 0.0035
Epoch 80/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8350 - accuracy: 0.0289 - val_loss: 3.9245 - val_accuracy: 0.0035
Epoch 81/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8350 - accuracy: 0.0289 - val_loss: 3.9249 - val_accuracy: 0.0035
Epoch 82/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8349 - accuracy: 0.0289 - val_loss: 3.9254 - val_accuracy: 0.0035
Epoch 83/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8349 - accuracy: 0.0289 - val_loss: 3.9257 - val_accuracy: 0.0035
Epoch 84/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8350 - accuracy: 0.0289 - val_loss: 3.9261 - val_accuracy: 0.0035
Epoch 85/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8349 - accuracy: 0.0289 - val_loss: 3.9264 - val_accuracy: 0.0035
Epoch 86/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8349 - accuracy: 0.0289 - val_loss: 3.9267 - val_accuracy: 0.0035
Epoch 87/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8349 - accuracy: 0.0289 - val_loss: 3.9270 - val_accuracy: 0.0035
Epoch 88/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8348 - accuracy: 0.0289 - val_loss: 3.9275 - val_accuracy: 0.0035
Epoch 89/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8348 - accuracy: 0.0289 - val_loss: 3.9278 - val_accuracy: 0.0035
Epoch 90/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8348 - accuracy: 0.0289 - val_loss: 3.9281 - val_accuracy: 0.0035
Epoch 91/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8348 - accuracy: 0.0289 - val_loss: 3.9285 - val_accuracy: 0.0035
Epoch 92/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8347 - accuracy: 0.0289 - val_loss: 3.9289 - val_accuracy: 0.0035
Epoch 93/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8348 - accuracy: 0.0289 - val_loss: 3.9293 - val_accuracy: 0.0035
Epoch 94/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8348 - accuracy: 0.0289 - val_loss: 3.9297 - val_accuracy: 0.0035
Epoch 95/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8347 - accuracy: 0.0289 - val_loss: 3.9300 - val_accuracy: 0.0035
Epoch 96/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8347 - accuracy: 0.0289 - val_loss: 3.9303 - val_accuracy: 0.0035
Epoch 97/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8347 - accuracy: 0.0289 - val_loss: 3.9305 - val_accuracy: 0.0035
Epoch 98/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8347 - accuracy: 0.0289 - val_loss: 3.9307 - val_accuracy: 0.0035
Epoch 99/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8347 - accuracy: 0.0289 - val_loss: 3.9311 - val_accuracy: 0.0035
Epoch 100/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8347 - accuracy: 0.0289 - val_loss: 3.9314 - val_accuracy: 0.0035
In [76]:
y_pred_5 = model_5.predict(X_test_tf)

9/9 [==============================] - 0s 2ms/step
In [77]:
model_5.evaluate(X_test_tf, y_test_tf)

9/9 [==============================] - 0s 2ms/step - loss: 3.9314 - accuracy: 0.0035
Out[77]:
[3.9313576221466064, 0.003546099178493023]
In [78]:
df_history_5 = pd.DataFrame(history_5.history)
In [79]:
fig = px.line(df_history_5, y=['loss', 'val_loss'], labels=['Loss', 'Validation Loss'])
fig.show()
In [ ]:

Let's explore predicted results¶

This wil indicate probablity of first preictions being in each class.

In [80]:
y_pred_1[0]
Out[80]:
array([4.8449532e-08, 3.5367210e-16, 7.6645368e-04, 3.1939714e-04,
       2.4672414e-04, 1.2412488e-07, 2.2907568e-07, 2.4982702e-04,
       1.4525743e-06, 3.7733134e-05, 6.2474800e-07, 2.2881359e-04,
       2.1208020e-08, 2.6287230e-13, 7.6839191e-10, 8.7677210e-05,
       3.6283085e-04, 4.8827951e-06, 1.6351011e-06, 8.1181475e-05,
       1.2000803e-05, 1.3543415e-05, 1.0420146e-03, 1.0558730e-13,
       6.4393108e-10, 4.0226284e-04, 1.3963463e-06, 1.0608255e-04,
       9.9502450e-01, 1.0040873e-09, 1.0059102e-05, 5.0157132e-07,
       1.9160314e-13, 9.2843658e-12, 3.5786619e-13, 8.6034364e-05,
       6.2172984e-07, 7.3428723e-08, 1.2153090e-04, 2.0742400e-04,
       5.7648699e-04, 5.2906084e-06, 1.8398772e-08, 2.7886523e-11,
       4.2623782e-07, 6.3658958e-09, 6.3064179e-11], dtype=float32)

This will get all prediction classes of Test dataset.

In [81]:
y_pred_1.argmax(axis=1)
Out[81]:
array([28, 46,  2, 29, 18, 32, 15, 10, 19, 31, 13,  7,  3, 21, 40, 33, 27,
       29, 22, 13, 33,  4, 31, 22,  8, 15,  0,  6, 34,  4, 13,  4, 30, 19,
       14, 41, 28, 28, 43, 20, 28, 42,  9, 18, 28,  1, 40, 24, 43, 35, 36,
       28, 31, 30, 12, 46,  4,  0, 12, 12, 32, 23,  6, 10, 13, 44,  8, 15,
       12, 32, 21, 25, 11, 22, 26, 23, 32,  8, 34, 14, 45, 33, 46, 17, 20,
       26,  8, 13, 11, 22, 33, 11, 15, 28,  8, 20, 33, 10, 37, 43, 11,  9,
       32, 33, 18, 30,  6, 21, 18, 21, 40, 33, 34, 36,  5, 17, 21, 37, 22,
       43, 41,  6, 35,  2,  1, 35,  9, 26, 40, 11, 44,  6, 10, 14, 43, 26,
       13,  2, 45, 12, 20, 42, 16, 23, 24, 11, 24, 17, 27,  9,  7, 37, 24,
       28, 23, 18, 21,  3, 14, 14, 23, 43, 41, 13, 16, 14,  9, 18, 32, 35,
        7, 42, 21, 11, 46, 32, 24, 27, 27, 46, 19, 10, 32,  9,  1, 44,  6,
       36, 27, 46, 36, 46, 43, 27,  1, 37, 45, 23, 38, 27, 19, 29, 23, 40,
        0, 37, 30,  7, 28, 44, 40, 41, 13,  0, 38, 14, 13, 18, 22, 18,  1,
       43, 13, 10, 27, 19, 12, 45, 42,  6, 42, 27, 40, 17, 26, 41, 40, 19,
        6,  6, 44, 32, 12, 45, 25, 16,  7, 39, 18, 32, 41, 46, 27, 21, 12,
       32, 43, 39, 38,  3, 18, 37, 23, 35, 23, 24, 37, 44, 14, 44, 24, 19,
       42, 20,  7, 11, 33,  6, 27,  3, 12, 46], dtype=int64)
In [82]:
# This is original result
y_test_tf
Out[82]:
<tf.Tensor: shape=(282,), dtype=float64, numpy=
array([28., 46.,  2., 29., 18., 32., 15., 10., 19., 31., 13.,  7.,  3.,
       21., 40., 33., 27., 29., 22., 13., 33.,  4., 31., 22.,  8., 15.,
        0.,  6., 34.,  4., 13.,  4., 30., 19., 14., 41., 28., 28., 43.,
       20., 28., 42.,  9., 18., 28.,  1., 40., 24., 43., 35., 36., 28.,
       31., 30., 12., 46.,  4.,  0., 12., 12., 32., 23.,  6., 10., 13.,
       44.,  8., 15., 12., 32., 21., 25., 11., 22., 26., 23., 32.,  8.,
       34., 14., 45., 33., 46., 17., 20., 26.,  8., 13., 11., 22., 33.,
       11., 15., 28.,  8., 20., 33., 10., 37., 43., 11.,  9., 32., 33.,
       18., 30.,  6., 21., 18., 21., 40., 33., 34., 36.,  5., 17., 21.,
       37., 22., 43., 41.,  6., 35.,  2.,  1., 35.,  9., 26., 40., 11.,
       44.,  6., 10., 14., 43., 26., 13.,  2., 45., 12., 20., 42., 16.,
       23., 24., 11., 24., 17., 27.,  9.,  7., 37., 24., 28., 23., 18.,
       21.,  3., 14., 14., 23., 43., 41., 13., 16., 14.,  9., 18., 32.,
       35.,  7., 42., 21., 11., 46., 32., 24., 27., 27., 46., 19., 10.,
       32.,  9.,  1., 44.,  6., 36., 27., 46., 36., 46., 43., 27.,  1.,
       37., 45., 23., 38., 27., 19., 29., 23., 40.,  0., 37., 30.,  7.,
       28., 44., 40., 41., 13.,  0., 38., 14., 13., 18., 22., 18.,  1.,
       43., 13., 10., 38., 19., 12., 45., 42.,  6., 42., 27., 40., 17.,
       26., 41., 40., 19.,  6.,  6., 44., 32., 12., 45., 25., 16.,  7.,
       39., 18., 32., 41., 46., 27., 21., 12., 32., 43., 39., 38.,  3.,
       18., 37., 23., 35., 23., 24., 37., 44., 14., 44., 24., 19., 42.,
       20.,  7., 11., 33.,  6., 27.,  3., 12., 46.])>
In [83]:
acc_1 = accuracy_score(y_test_tf, y_pred_1.argmax(axis=1))
In [84]:
acc_2 = accuracy_score(y_test_tf, y_pred_2.argmax(axis=1))
In [85]:
acc_3 = accuracy_score(y_test_tf, y_pred_3.argmax(axis=1))
In [86]:
acc_4 = accuracy_score(y_test_tf, y_pred_4.argmax(axis=1))
In [87]:
acc_5 = accuracy_score(y_test_tf, y_pred_5.argmax(axis=1))

Let's plot accuracy for all 5 model

In [88]:
plt.bar(x=["Model 1", "Model 2", "Model 3", "Model 4", "Model 5"], height=[acc_1, acc_2, acc_3, acc_4, acc_5])
Out[88]:
<BarContainer object of 5 artists>

Looks like we should move forward with Model 1.¶

Before that, let's explore results through confusion matrix

In [89]:
import itertools
import matplotlib.pyplot as plt
import numpy as np
from sklearn.metrics import confusion_matrix


# Our function needs a different name to sklearn's plot_confusion_matrix
def make_confusion_matrix(y_true, y_pred, classes=None, figsize=(10, 10), text_size=15, norm=False, savefig=False):
    """Makes a labelled confusion matrix comparing predictions and ground truth labels.
    If classes is passed, confusion matrix will be labelled, if not, integer class values
    will be used.
    Args:
      y_true: Array of truth labels (must be same shape as y_pred).
      y_pred: Array of predicted labels (must be same shape as y_true).
      classes: Array of class labels (e.g. string form). If `None`, integer labels are used.
      figsize: Size of output figure (default=(10, 10)).
      text_size: Size of output figure text (default=15).
      norm: normalize values or not (default=False).
      savefig: save confusion matrix to file (default=False).
    Returns:
      A labelled confusion matrix plot comparing y_true and y_pred.
    Example usage:
      make_confusion_matrix(y_true=test_labels, # ground truth test labels
                            y_pred=y_preds, # predicted labels
                            classes=class_names, # array of class label names
                            figsize=(15, 15),
                            text_size=10)
    """
    # Create the confustion matrix
    cm = confusion_matrix(y_true, y_pred)
    cm_norm = cm.astype("float") / cm.sum(axis=1)[:, np.newaxis]  # normalize it
    n_classes = cm.shape[0]  # find the number of classes we're dealing with

    # Plot the figure and make it pretty
    fig, ax = plt.subplots(figsize=figsize)
    cax = ax.matshow(cm, cmap=plt.cm.Blues)  # colors will represent how 'correct' a class is, darker == better
    fig.colorbar(cax)

    # Are there a list of classes?
    if classes:
        labels = classes
    else:
        labels = np.arange(cm.shape[0])

    # Label the axes
    ax.set(title="Confusion Matrix",
           xlabel="Predicted label",
           ylabel="True label",
           xticks=np.arange(n_classes),  # create enough axis slots for each class
           yticks=np.arange(n_classes),
           xticklabels=labels,  # axes will labeled with class names (if they exist) or ints
           yticklabels=labels)

    # Make x-axis labels appear on bottom
    ax.xaxis.set_label_position("bottom")
    ax.xaxis.tick_bottom()

    # Set the threshold for different colors
    threshold = (cm.max() + cm.min()) / 2.

    # Plot the text on each cell
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        if norm:
            plt.text(j, i, f"{cm[i, j]} ({cm_norm[i, j] * 100:.1f}%)",
                     horizontalalignment="center",
                     color="white" if cm[i, j] > threshold else "black",
                     size=text_size)
        else:
            plt.text(j, i, f"{cm[i, j]}",
                     horizontalalignment="center",
                     color="white" if cm[i, j] > threshold else "black",
                     size=text_size)

    # Save the figure to the current working directory
    if savefig:
        fig.savefig("confusion_matrix.png")

Confusion Matrix: Model 1¶

It seems more organized and predicted values are mostly true.

In [90]:
make_confusion_matrix(y_test_tf, y_pred_1.argmax(axis=1), figsize=(30, 30), text_size=10, classes=list(ordinalencoder.categories_[0]))

Confusion Matrix: Model 2¶

This output seems to have bias and most of the predictions happeed in two classes.

In [91]:
make_confusion_matrix(y_test_tf, y_pred_2.argmax(axis=1), figsize=(30, 30), text_size=10, classes=list(ordinalencoder.categories_[0]))

Confusion Matrix: Model 3¶

This model's prediction seems all over the place without any clear pattern.

In [92]:
make_confusion_matrix(y_test_tf, y_pred_3.argmax(axis=1), figsize=(30, 30), text_size=10, classes=list(ordinalencoder.categories_[0]))

Confusion Matrix: Model 4¶

This model's result is also similar to Model 3

In [93]:
make_confusion_matrix(y_test_tf, y_pred_4.argmax(axis=1), figsize=(30, 30), text_size=10, classes=list(ordinalencoder.categories_[0]))

Confusion Matrix: Model 5¶

This model seems to think every character is from Cerea.

In [94]:
make_confusion_matrix(y_test_tf, y_pred_5.argmax(axis=1), figsize=(30, 30), text_size=10, classes=list(ordinalencoder.categories_[0]))

Since we got expected output from Model 1. Let's explore it a bit by changing learning rate

Model 1_1 - Learning Rate - 0.1¶

In [95]:
model_1_1 = tf.keras.Sequential([
    layers.Input(shape=(X_train.shape[1],)),
    layers.Dense(256, activation="relu"),
    layers.Dense(47, activation="softmax")
])

model_1_1.compile(loss="sparse_categorical_crossentropy", optimizer=optimizers.Adam(learning_rate=0.1), metrics=["accuracy"])
In [96]:
history_1_1 = model_1_1.fit(X_train_tf, y_train_tf, epochs=100, validation_data=(X_test, y_test))

Epoch 1/100
21/21 [==============================] - 1s 8ms/step - loss: 53.6355 - accuracy: 0.0182 - val_loss: 3.9202 - val_accuracy: 0.0071
Epoch 2/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8961 - accuracy: 0.0122 - val_loss: 3.9395 - val_accuracy: 0.0071
Epoch 3/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8687 - accuracy: 0.0213 - val_loss: 3.9444 - val_accuracy: 0.0071
Epoch 4/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8581 - accuracy: 0.0258 - val_loss: 3.9426 - val_accuracy: 0.0035
Epoch 5/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8605 - accuracy: 0.0122 - val_loss: 3.9452 - val_accuracy: 0.0035
Epoch 6/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8567 - accuracy: 0.0228 - val_loss: 3.9431 - val_accuracy: 0.0106
Epoch 7/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8678 - accuracy: 0.0228 - val_loss: 3.9531 - val_accuracy: 0.0035
Epoch 8/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8566 - accuracy: 0.0228 - val_loss: 3.9449 - val_accuracy: 0.0106
Epoch 9/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8574 - accuracy: 0.0258 - val_loss: 3.9453 - val_accuracy: 0.0035
Epoch 10/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8628 - accuracy: 0.0122 - val_loss: 3.9482 - val_accuracy: 0.0071
Epoch 11/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8594 - accuracy: 0.0243 - val_loss: 3.9528 - val_accuracy: 0.0106
Epoch 12/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8582 - accuracy: 0.0213 - val_loss: 3.9539 - val_accuracy: 0.0035
Epoch 13/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8600 - accuracy: 0.0228 - val_loss: 3.9523 - val_accuracy: 0.0035
Epoch 14/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8582 - accuracy: 0.0137 - val_loss: 3.9338 - val_accuracy: 0.0106
Epoch 15/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8605 - accuracy: 0.0167 - val_loss: 3.9658 - val_accuracy: 0.0106
Epoch 16/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8561 - accuracy: 0.0319 - val_loss: 3.9517 - val_accuracy: 0.0035
Epoch 17/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8604 - accuracy: 0.0167 - val_loss: 3.9411 - val_accuracy: 0.0071
Epoch 18/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8624 - accuracy: 0.0213 - val_loss: 3.9519 - val_accuracy: 0.0071
Epoch 19/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8595 - accuracy: 0.0274 - val_loss: 3.9433 - val_accuracy: 0.0035
Epoch 20/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8669 - accuracy: 0.0274 - val_loss: 3.9622 - val_accuracy: 0.0106
Epoch 21/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8598 - accuracy: 0.0106 - val_loss: 3.9476 - val_accuracy: 0.0142
Epoch 22/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8565 - accuracy: 0.0274 - val_loss: 3.9422 - val_accuracy: 0.0106
Epoch 23/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8604 - accuracy: 0.0274 - val_loss: 3.9476 - val_accuracy: 0.0035
Epoch 24/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8600 - accuracy: 0.0243 - val_loss: 3.9494 - val_accuracy: 0.0071
Epoch 25/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8583 - accuracy: 0.0243 - val_loss: 3.9563 - val_accuracy: 0.0035
Epoch 26/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8601 - accuracy: 0.0243 - val_loss: 3.9355 - val_accuracy: 0.0106
Epoch 27/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8561 - accuracy: 0.0228 - val_loss: 3.9466 - val_accuracy: 0.0071
Epoch 28/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8631 - accuracy: 0.0274 - val_loss: 3.9624 - val_accuracy: 0.0035
Epoch 29/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8639 - accuracy: 0.0228 - val_loss: 3.9558 - val_accuracy: 0.0071
Epoch 30/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8586 - accuracy: 0.0213 - val_loss: 3.9381 - val_accuracy: 0.0106
Epoch 31/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8642 - accuracy: 0.0304 - val_loss: 3.9563 - val_accuracy: 0.0035
Epoch 32/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8599 - accuracy: 0.0198 - val_loss: 3.9525 - val_accuracy: 0.0035
Epoch 33/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8675 - accuracy: 0.0213 - val_loss: 3.9478 - val_accuracy: 0.0071
Epoch 34/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8623 - accuracy: 0.0137 - val_loss: 3.9496 - val_accuracy: 0.0035
Epoch 35/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8588 - accuracy: 0.0167 - val_loss: 3.9499 - val_accuracy: 0.0071
Epoch 36/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8638 - accuracy: 0.0167 - val_loss: 3.9555 - val_accuracy: 0.0071
Epoch 37/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8617 - accuracy: 0.0274 - val_loss: 3.9470 - val_accuracy: 0.0035
Epoch 38/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8656 - accuracy: 0.0198 - val_loss: 3.9422 - val_accuracy: 0.0071
Epoch 39/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8643 - accuracy: 0.0243 - val_loss: 3.9567 - val_accuracy: 0.0035
Epoch 40/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8619 - accuracy: 0.0213 - val_loss: 3.9471 - val_accuracy: 0.0035
Epoch 41/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8599 - accuracy: 0.0167 - val_loss: 3.9593 - val_accuracy: 0.0071
Epoch 42/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8643 - accuracy: 0.0152 - val_loss: 3.9307 - val_accuracy: 0.0035
Epoch 43/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8713 - accuracy: 0.0213 - val_loss: 3.9613 - val_accuracy: 0.0142
Epoch 44/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8652 - accuracy: 0.0198 - val_loss: 3.9382 - val_accuracy: 0.0106
Epoch 45/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8575 - accuracy: 0.0243 - val_loss: 3.9572 - val_accuracy: 0.0106
Epoch 46/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8542 - accuracy: 0.0243 - val_loss: 3.9486 - val_accuracy: 0.0035
Epoch 47/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8624 - accuracy: 0.0213 - val_loss: 3.9454 - val_accuracy: 0.0142
Epoch 48/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8618 - accuracy: 0.0167 - val_loss: 3.9538 - val_accuracy: 0.0035
Epoch 49/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8592 - accuracy: 0.0152 - val_loss: 3.9499 - val_accuracy: 0.0035
Epoch 50/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8624 - accuracy: 0.0243 - val_loss: 3.9573 - val_accuracy: 0.0071
Epoch 51/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8600 - accuracy: 0.0228 - val_loss: 3.9375 - val_accuracy: 0.0035
Epoch 52/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8587 - accuracy: 0.0152 - val_loss: 3.9472 - val_accuracy: 0.0106
Epoch 53/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8629 - accuracy: 0.0076 - val_loss: 3.9530 - val_accuracy: 0.0035
Epoch 54/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8608 - accuracy: 0.0213 - val_loss: 3.9669 - val_accuracy: 0.0106
Epoch 55/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8585 - accuracy: 0.0243 - val_loss: 3.9443 - val_accuracy: 0.0106
Epoch 56/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8599 - accuracy: 0.0228 - val_loss: 3.9470 - val_accuracy: 0.0071
Epoch 57/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8575 - accuracy: 0.0274 - val_loss: 3.9529 - val_accuracy: 0.0035
Epoch 58/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8546 - accuracy: 0.0258 - val_loss: 3.9454 - val_accuracy: 0.0071
Epoch 59/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8560 - accuracy: 0.0198 - val_loss: 3.9519 - val_accuracy: 0.0106
Epoch 60/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8630 - accuracy: 0.0258 - val_loss: 3.9707 - val_accuracy: 0.0035
Epoch 61/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8606 - accuracy: 0.0213 - val_loss: 3.9417 - val_accuracy: 0.0071
Epoch 62/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8615 - accuracy: 0.0198 - val_loss: 3.9519 - val_accuracy: 0.0142
Epoch 63/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8588 - accuracy: 0.0198 - val_loss: 3.9616 - val_accuracy: 0.0035
Epoch 64/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8574 - accuracy: 0.0167 - val_loss: 3.9396 - val_accuracy: 0.0071
Epoch 65/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8583 - accuracy: 0.0198 - val_loss: 3.9582 - val_accuracy: 0.0035
Epoch 66/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8568 - accuracy: 0.0152 - val_loss: 3.9471 - val_accuracy: 0.0106
Epoch 67/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8640 - accuracy: 0.0289 - val_loss: 3.9629 - val_accuracy: 0.0035
Epoch 68/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8693 - accuracy: 0.0243 - val_loss: 3.9379 - val_accuracy: 0.0071
Epoch 69/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8619 - accuracy: 0.0198 - val_loss: 3.9534 - val_accuracy: 0.0071
Epoch 70/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8649 - accuracy: 0.0152 - val_loss: 3.9517 - val_accuracy: 0.0071
Epoch 71/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8626 - accuracy: 0.0167 - val_loss: 3.9363 - val_accuracy: 0.0035
Epoch 72/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8588 - accuracy: 0.0198 - val_loss: 3.9397 - val_accuracy: 0.0035
Epoch 73/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8597 - accuracy: 0.0289 - val_loss: 3.9692 - val_accuracy: 0.0106
Epoch 74/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8636 - accuracy: 0.0258 - val_loss: 3.9348 - val_accuracy: 0.0142
Epoch 75/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8642 - accuracy: 0.0243 - val_loss: 3.9492 - val_accuracy: 0.0035
Epoch 76/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8625 - accuracy: 0.0198 - val_loss: 3.9669 - val_accuracy: 0.0106
Epoch 77/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8667 - accuracy: 0.0198 - val_loss: 3.9400 - val_accuracy: 0.0106
Epoch 78/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8652 - accuracy: 0.0228 - val_loss: 3.9542 - val_accuracy: 0.0071
Epoch 79/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8614 - accuracy: 0.0137 - val_loss: 3.9426 - val_accuracy: 0.0106
Epoch 80/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8634 - accuracy: 0.0182 - val_loss: 3.9423 - val_accuracy: 0.0035
Epoch 81/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8634 - accuracy: 0.0106 - val_loss: 3.9531 - val_accuracy: 0.0106
Epoch 82/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8724 - accuracy: 0.0274 - val_loss: 3.9495 - val_accuracy: 0.0071
Epoch 83/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8591 - accuracy: 0.0258 - val_loss: 3.9594 - val_accuracy: 0.0035
Epoch 84/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8625 - accuracy: 0.0228 - val_loss: 3.9427 - val_accuracy: 0.0106
Epoch 85/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8615 - accuracy: 0.0182 - val_loss: 3.9609 - val_accuracy: 0.0035
Epoch 86/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8648 - accuracy: 0.0167 - val_loss: 3.9391 - val_accuracy: 0.0071
Epoch 87/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8582 - accuracy: 0.0137 - val_loss: 3.9451 - val_accuracy: 0.0106
Epoch 88/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8570 - accuracy: 0.0198 - val_loss: 3.9507 - val_accuracy: 0.0035
Epoch 89/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8659 - accuracy: 0.0167 - val_loss: 3.9547 - val_accuracy: 0.0071
Epoch 90/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8668 - accuracy: 0.0258 - val_loss: 3.9539 - val_accuracy: 0.0035
Epoch 91/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8648 - accuracy: 0.0122 - val_loss: 3.9342 - val_accuracy: 0.0106
Epoch 92/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8616 - accuracy: 0.0182 - val_loss: 3.9546 - val_accuracy: 0.0071
Epoch 93/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8619 - accuracy: 0.0152 - val_loss: 3.9498 - val_accuracy: 0.0035
Epoch 94/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8610 - accuracy: 0.0182 - val_loss: 3.9435 - val_accuracy: 0.0035
Epoch 95/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8632 - accuracy: 0.0289 - val_loss: 3.9537 - val_accuracy: 0.0071
Epoch 96/100
21/21 [==============================] - 0s 4ms/step - loss: 3.8627 - accuracy: 0.0167 - val_loss: 3.9493 - val_accuracy: 0.0071
Epoch 97/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8660 - accuracy: 0.0243 - val_loss: 3.9548 - val_accuracy: 0.0071
Epoch 98/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8615 - accuracy: 0.0182 - val_loss: 3.9417 - val_accuracy: 0.0035
Epoch 99/100
21/21 [==============================] - 0s 2ms/step - loss: 3.8581 - accuracy: 0.0243 - val_loss: 3.9537 - val_accuracy: 0.0071
Epoch 100/100
21/21 [==============================] - 0s 3ms/step - loss: 3.8598 - accuracy: 0.0213 - val_loss: 3.9460 - val_accuracy: 0.0106
In [97]:
y_pred_1_1 = model_1.predict(X_test_tf)

9/9 [==============================] - 0s 0s/step
In [98]:
model_1_1.evaluate(X_test_tf, y_test_tf)

9/9 [==============================] - 0s 2ms/step - loss: 3.9460 - accuracy: 0.0106
Out[98]:
[3.9459519386291504, 0.010638297535479069]
In [99]:
plot_loss_curves(history_1_1)

As expected, since Model has very high learning rate. We hit new bottom for accuracy.

Model 1_2 - Learning rate - 0.01¶

In [100]:
model_1_2 = tf.keras.Sequential([
    layers.Input(shape=(X_train.shape[1],)),
    layers.Dense(256, activation="relu"),
    layers.Dense(47, activation="softmax")
])

model_1_2.compile(loss="sparse_categorical_crossentropy", optimizer=optimizers.Adam(learning_rate=0.01), metrics=["accuracy"])
In [101]:
history_1_2 = model_1_2.fit(X_train_tf, y_train_tf, epochs=100, validation_data=(X_test, y_test))

Epoch 1/100
21/21 [==============================] - 1s 8ms/step - loss: 28.1268 - accuracy: 0.0471 - val_loss: 9.9578 - val_accuracy: 0.0957
Epoch 2/100
21/21 [==============================] - 0s 3ms/step - loss: 5.6975 - accuracy: 0.1064 - val_loss: 3.3248 - val_accuracy: 0.0922
Epoch 3/100
21/21 [==============================] - 0s 3ms/step - loss: 3.0338 - accuracy: 0.1702 - val_loss: 2.7307 - val_accuracy: 0.2589
Epoch 4/100
21/21 [==============================] - 0s 3ms/step - loss: 2.3799 - accuracy: 0.4271 - val_loss: 2.0575 - val_accuracy: 0.5213
Epoch 5/100
21/21 [==============================] - 0s 3ms/step - loss: 1.6325 - accuracy: 0.6900 - val_loss: 1.3633 - val_accuracy: 0.7482
Epoch 6/100
21/21 [==============================] - 0s 3ms/step - loss: 1.0342 - accuracy: 0.8450 - val_loss: 0.7562 - val_accuracy: 0.9184
Epoch 7/100
21/21 [==============================] - 0s 3ms/step - loss: 0.5954 - accuracy: 0.9179 - val_loss: 0.5698 - val_accuracy: 0.9184
Epoch 8/100
21/21 [==============================] - 0s 3ms/step - loss: 0.3564 - accuracy: 0.9468 - val_loss: 0.3329 - val_accuracy: 0.9504
Epoch 9/100
21/21 [==============================] - 0s 3ms/step - loss: 0.2401 - accuracy: 0.9666 - val_loss: 0.3125 - val_accuracy: 0.9539
Epoch 10/100
21/21 [==============================] - 0s 3ms/step - loss: 0.1744 - accuracy: 0.9802 - val_loss: 0.1963 - val_accuracy: 0.9681
Epoch 11/100
21/21 [==============================] - 0s 5ms/step - loss: 0.1265 - accuracy: 0.9818 - val_loss: 0.1153 - val_accuracy: 0.9858
Epoch 12/100
21/21 [==============================] - 0s 5ms/step - loss: 0.0973 - accuracy: 0.9863 - val_loss: 0.1302 - val_accuracy: 0.9681
Epoch 13/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0886 - accuracy: 0.9894 - val_loss: 0.1073 - val_accuracy: 0.9894
Epoch 14/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0761 - accuracy: 0.9939 - val_loss: 0.1004 - val_accuracy: 0.9752
Epoch 15/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0573 - accuracy: 0.9909 - val_loss: 0.1056 - val_accuracy: 0.9894
Epoch 16/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0612 - accuracy: 0.9954 - val_loss: 0.0754 - val_accuracy: 0.9929
Epoch 17/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0501 - accuracy: 0.9939 - val_loss: 0.0679 - val_accuracy: 0.9894
Epoch 18/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0367 - accuracy: 0.9954 - val_loss: 0.0657 - val_accuracy: 0.9894
Epoch 19/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0418 - accuracy: 0.9939 - val_loss: 0.0563 - val_accuracy: 0.9929
Epoch 20/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0268 - accuracy: 0.9970 - val_loss: 0.0589 - val_accuracy: 0.9929
Epoch 21/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0307 - accuracy: 0.9954 - val_loss: 0.0539 - val_accuracy: 0.9894
Epoch 22/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0278 - accuracy: 0.9954 - val_loss: 0.0577 - val_accuracy: 0.9858
Epoch 23/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0209 - accuracy: 0.9954 - val_loss: 0.0536 - val_accuracy: 0.9929
Epoch 24/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0198 - accuracy: 0.9985 - val_loss: 0.0449 - val_accuracy: 0.9929
Epoch 25/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0151 - accuracy: 1.0000 - val_loss: 0.0468 - val_accuracy: 0.9929
Epoch 26/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0120 - accuracy: 1.0000 - val_loss: 0.0551 - val_accuracy: 0.9929
Epoch 27/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0129 - accuracy: 0.9985 - val_loss: 0.0487 - val_accuracy: 0.9929
Epoch 28/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0103 - accuracy: 1.0000 - val_loss: 0.0510 - val_accuracy: 0.9929
Epoch 29/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0096 - accuracy: 0.9985 - val_loss: 0.0465 - val_accuracy: 0.9929
Epoch 30/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0126 - accuracy: 0.9970 - val_loss: 0.0503 - val_accuracy: 0.9858
Epoch 31/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0124 - accuracy: 0.9985 - val_loss: 0.0491 - val_accuracy: 0.9929
Epoch 32/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0152 - accuracy: 0.9985 - val_loss: 0.0483 - val_accuracy: 0.9823
Epoch 33/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0116 - accuracy: 0.9985 - val_loss: 0.0573 - val_accuracy: 0.9929
Epoch 34/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0073 - accuracy: 1.0000 - val_loss: 0.0575 - val_accuracy: 0.9929
Epoch 35/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0072 - accuracy: 1.0000 - val_loss: 0.0480 - val_accuracy: 0.9929
Epoch 36/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0054 - accuracy: 1.0000 - val_loss: 0.0492 - val_accuracy: 0.9929
Epoch 37/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0043 - accuracy: 1.0000 - val_loss: 0.0489 - val_accuracy: 0.9929
Epoch 38/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0040 - accuracy: 1.0000 - val_loss: 0.0472 - val_accuracy: 0.9929
Epoch 39/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0037 - accuracy: 1.0000 - val_loss: 0.0487 - val_accuracy: 0.9929
Epoch 40/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0034 - accuracy: 1.0000 - val_loss: 0.0500 - val_accuracy: 0.9929
Epoch 41/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0034 - accuracy: 1.0000 - val_loss: 0.0472 - val_accuracy: 0.9929
Epoch 42/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0034 - accuracy: 1.0000 - val_loss: 0.0481 - val_accuracy: 0.9929
Epoch 43/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0031 - accuracy: 1.0000 - val_loss: 0.0477 - val_accuracy: 0.9929
Epoch 44/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0030 - accuracy: 1.0000 - val_loss: 0.0481 - val_accuracy: 0.9929
Epoch 45/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0028 - accuracy: 1.0000 - val_loss: 0.0469 - val_accuracy: 0.9929
Epoch 46/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0028 - accuracy: 1.0000 - val_loss: 0.0473 - val_accuracy: 0.9929
Epoch 47/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0025 - accuracy: 1.0000 - val_loss: 0.0490 - val_accuracy: 0.9929
Epoch 48/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0025 - accuracy: 1.0000 - val_loss: 0.0471 - val_accuracy: 0.9929
Epoch 49/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0024 - accuracy: 1.0000 - val_loss: 0.0485 - val_accuracy: 0.9929
Epoch 50/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0023 - accuracy: 1.0000 - val_loss: 0.0484 - val_accuracy: 0.9929
Epoch 51/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0022 - accuracy: 1.0000 - val_loss: 0.0512 - val_accuracy: 0.9929
Epoch 52/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0021 - accuracy: 1.0000 - val_loss: 0.0488 - val_accuracy: 0.9929
Epoch 53/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0022 - accuracy: 1.0000 - val_loss: 0.0487 - val_accuracy: 0.9929
Epoch 54/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0019 - accuracy: 1.0000 - val_loss: 0.0489 - val_accuracy: 0.9929
Epoch 55/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0019 - accuracy: 1.0000 - val_loss: 0.0476 - val_accuracy: 0.9929
Epoch 56/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0018 - accuracy: 1.0000 - val_loss: 0.0504 - val_accuracy: 0.9929
Epoch 57/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0018 - accuracy: 1.0000 - val_loss: 0.0486 - val_accuracy: 0.9929
Epoch 58/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0017 - accuracy: 1.0000 - val_loss: 0.0487 - val_accuracy: 0.9929
Epoch 59/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0016 - accuracy: 1.0000 - val_loss: 0.0487 - val_accuracy: 0.9929
Epoch 60/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0016 - accuracy: 1.0000 - val_loss: 0.0486 - val_accuracy: 0.9929
Epoch 61/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0016 - accuracy: 1.0000 - val_loss: 0.0492 - val_accuracy: 0.9929
Epoch 62/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0015 - accuracy: 1.0000 - val_loss: 0.0489 - val_accuracy: 0.9929
Epoch 63/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0015 - accuracy: 1.0000 - val_loss: 0.0508 - val_accuracy: 0.9929
Epoch 64/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0014 - accuracy: 1.0000 - val_loss: 0.0505 - val_accuracy: 0.9929
Epoch 65/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0014 - accuracy: 1.0000 - val_loss: 0.0504 - val_accuracy: 0.9929
Epoch 66/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0013 - accuracy: 1.0000 - val_loss: 0.0500 - val_accuracy: 0.9929
Epoch 67/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0013 - accuracy: 1.0000 - val_loss: 0.0502 - val_accuracy: 0.9929
Epoch 68/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0012 - accuracy: 1.0000 - val_loss: 0.0500 - val_accuracy: 0.9929
Epoch 69/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0012 - accuracy: 1.0000 - val_loss: 0.0518 - val_accuracy: 0.9929
Epoch 70/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0012 - accuracy: 1.0000 - val_loss: 0.0503 - val_accuracy: 0.9929
Epoch 71/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0011 - accuracy: 1.0000 - val_loss: 0.0510 - val_accuracy: 0.9929
Epoch 72/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0011 - accuracy: 1.0000 - val_loss: 0.0494 - val_accuracy: 0.9929
Epoch 73/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0011 - accuracy: 1.0000 - val_loss: 0.0508 - val_accuracy: 0.9929
Epoch 74/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0011 - accuracy: 1.0000 - val_loss: 0.0505 - val_accuracy: 0.9929
Epoch 75/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0010 - accuracy: 1.0000 - val_loss: 0.0506 - val_accuracy: 0.9929
Epoch 76/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0010 - accuracy: 1.0000 - val_loss: 0.0509 - val_accuracy: 0.9929
Epoch 77/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0010 - accuracy: 1.0000 - val_loss: 0.0500 - val_accuracy: 0.9929
Epoch 78/100
21/21 [==============================] - 0s 3ms/step - loss: 9.3633e-04 - accuracy: 1.0000 - val_loss: 0.0511 - val_accuracy: 0.9929
Epoch 79/100
21/21 [==============================] - 0s 3ms/step - loss: 9.2529e-04 - accuracy: 1.0000 - val_loss: 0.0506 - val_accuracy: 0.9929
Epoch 80/100
21/21 [==============================] - 0s 3ms/step - loss: 8.8564e-04 - accuracy: 1.0000 - val_loss: 0.0522 - val_accuracy: 0.9929
Epoch 81/100
21/21 [==============================] - 0s 3ms/step - loss: 8.7344e-04 - accuracy: 1.0000 - val_loss: 0.0505 - val_accuracy: 0.9929
Epoch 82/100
21/21 [==============================] - 0s 3ms/step - loss: 8.6250e-04 - accuracy: 1.0000 - val_loss: 0.0510 - val_accuracy: 0.9929
Epoch 83/100
21/21 [==============================] - 0s 2ms/step - loss: 8.3301e-04 - accuracy: 1.0000 - val_loss: 0.0516 - val_accuracy: 0.9929
Epoch 84/100
21/21 [==============================] - 0s 3ms/step - loss: 8.0602e-04 - accuracy: 1.0000 - val_loss: 0.0514 - val_accuracy: 0.9929
Epoch 85/100
21/21 [==============================] - 0s 3ms/step - loss: 7.8845e-04 - accuracy: 1.0000 - val_loss: 0.0519 - val_accuracy: 0.9929
Epoch 86/100
21/21 [==============================] - 0s 3ms/step - loss: 7.7483e-04 - accuracy: 1.0000 - val_loss: 0.0515 - val_accuracy: 0.9929
Epoch 87/100
21/21 [==============================] - 0s 3ms/step - loss: 7.6943e-04 - accuracy: 1.0000 - val_loss: 0.0522 - val_accuracy: 0.9929
Epoch 88/100
21/21 [==============================] - 0s 2ms/step - loss: 7.3379e-04 - accuracy: 1.0000 - val_loss: 0.0514 - val_accuracy: 0.9929
Epoch 89/100
21/21 [==============================] - 0s 3ms/step - loss: 7.1255e-04 - accuracy: 1.0000 - val_loss: 0.0518 - val_accuracy: 0.9929
Epoch 90/100
21/21 [==============================] - 0s 2ms/step - loss: 6.9511e-04 - accuracy: 1.0000 - val_loss: 0.0525 - val_accuracy: 0.9929
Epoch 91/100
21/21 [==============================] - 0s 2ms/step - loss: 6.7389e-04 - accuracy: 1.0000 - val_loss: 0.0522 - val_accuracy: 0.9929
Epoch 92/100
21/21 [==============================] - 0s 3ms/step - loss: 6.7589e-04 - accuracy: 1.0000 - val_loss: 0.0523 - val_accuracy: 0.9929
Epoch 93/100
21/21 [==============================] - 0s 3ms/step - loss: 6.4772e-04 - accuracy: 1.0000 - val_loss: 0.0516 - val_accuracy: 0.9929
Epoch 94/100
21/21 [==============================] - 0s 2ms/step - loss: 6.4342e-04 - accuracy: 1.0000 - val_loss: 0.0517 - val_accuracy: 0.9929
Epoch 95/100
21/21 [==============================] - 0s 2ms/step - loss: 6.2847e-04 - accuracy: 1.0000 - val_loss: 0.0532 - val_accuracy: 0.9929
Epoch 96/100
21/21 [==============================] - 0s 2ms/step - loss: 6.0936e-04 - accuracy: 1.0000 - val_loss: 0.0524 - val_accuracy: 0.9929
Epoch 97/100
21/21 [==============================] - 0s 3ms/step - loss: 5.9772e-04 - accuracy: 1.0000 - val_loss: 0.0524 - val_accuracy: 0.9929
Epoch 98/100
21/21 [==============================] - 0s 3ms/step - loss: 5.8020e-04 - accuracy: 1.0000 - val_loss: 0.0524 - val_accuracy: 0.9929
Epoch 99/100
21/21 [==============================] - 0s 3ms/step - loss: 5.7273e-04 - accuracy: 1.0000 - val_loss: 0.0524 - val_accuracy: 0.9929
Epoch 100/100
21/21 [==============================] - 0s 3ms/step - loss: 5.5205e-04 - accuracy: 1.0000 - val_loss: 0.0528 - val_accuracy: 0.9929
In [102]:
y_pred_1_2 = model_1_2.predict(X_test_tf)

9/9 [==============================] - 0s 0s/step
In [103]:
model_1_2.evaluate(X_test_tf, y_test_tf)

9/9 [==============================] - 0s 2ms/step - loss: 0.0528 - accuracy: 0.9929
Out[103]:
[0.052771955728530884, 0.9929078221321106]
In [104]:
plot_loss_curves(history_1_2)

Now this result seems more align with what we expecting.

Let's see if we can improve it by further reducing learning rate.

Model 1_3 - Learning Rate - 0.001¶

In [105]:
model_1_3 = tf.keras.Sequential([
    layers.Input(shape=(X_train.shape[1],)),
    layers.Dense(256, activation="relu"),
    layers.Dense(47, activation="softmax")
])

model_1_3.compile(loss="sparse_categorical_crossentropy", optimizer=optimizers.Adam(learning_rate=0.001), metrics=["accuracy"])
In [106]:
history_1_3 = model_1_3.fit(X_train_tf, y_train_tf, epochs=100, validation_data=(X_test, y_test))

Epoch 1/100
21/21 [==============================] - 1s 10ms/step - loss: 21.4124 - accuracy: 0.0350 - val_loss: 8.0968 - val_accuracy: 0.0142
Epoch 2/100
21/21 [==============================] - 0s 3ms/step - loss: 6.0075 - accuracy: 0.0638 - val_loss: 4.4360 - val_accuracy: 0.1028
Epoch 3/100
21/21 [==============================] - 0s 3ms/step - loss: 3.5829 - accuracy: 0.2143 - val_loss: 3.0788 - val_accuracy: 0.2695
Epoch 4/100
21/21 [==============================] - 0s 3ms/step - loss: 2.7309 - accuracy: 0.3632 - val_loss: 2.6305 - val_accuracy: 0.2979
Epoch 5/100
21/21 [==============================] - 0s 4ms/step - loss: 2.1874 - accuracy: 0.4985 - val_loss: 2.0552 - val_accuracy: 0.5816
Epoch 6/100
21/21 [==============================] - 0s 4ms/step - loss: 1.8488 - accuracy: 0.6185 - val_loss: 1.8536 - val_accuracy: 0.6277
Epoch 7/100
21/21 [==============================] - 0s 4ms/step - loss: 1.4629 - accuracy: 0.7492 - val_loss: 1.4397 - val_accuracy: 0.7872
Epoch 8/100
21/21 [==============================] - 0s 3ms/step - loss: 1.2706 - accuracy: 0.8207 - val_loss: 1.3079 - val_accuracy: 0.7270
Epoch 9/100
21/21 [==============================] - 0s 3ms/step - loss: 1.1388 - accuracy: 0.8055 - val_loss: 1.1291 - val_accuracy: 0.7624
Epoch 10/100
21/21 [==============================] - 0s 2ms/step - loss: 1.0247 - accuracy: 0.8359 - val_loss: 0.9254 - val_accuracy: 0.8688
Epoch 11/100
21/21 [==============================] - 0s 2ms/step - loss: 0.7912 - accuracy: 0.9119 - val_loss: 0.7445 - val_accuracy: 0.9397
Epoch 12/100
21/21 [==============================] - 0s 2ms/step - loss: 0.6409 - accuracy: 0.9362 - val_loss: 0.6798 - val_accuracy: 0.9291
Epoch 13/100
21/21 [==============================] - 0s 2ms/step - loss: 0.5822 - accuracy: 0.9331 - val_loss: 0.6345 - val_accuracy: 0.9078
Epoch 14/100
21/21 [==============================] - 0s 2ms/step - loss: 0.5288 - accuracy: 0.9316 - val_loss: 0.5047 - val_accuracy: 0.9574
Epoch 15/100
21/21 [==============================] - 0s 3ms/step - loss: 0.4193 - accuracy: 0.9635 - val_loss: 0.4281 - val_accuracy: 0.9504
Epoch 16/100
21/21 [==============================] - 0s 3ms/step - loss: 0.3703 - accuracy: 0.9635 - val_loss: 0.4067 - val_accuracy: 0.9574
Epoch 17/100
21/21 [==============================] - 0s 2ms/step - loss: 0.3314 - accuracy: 0.9696 - val_loss: 0.3649 - val_accuracy: 0.9716
Epoch 18/100
21/21 [==============================] - 0s 3ms/step - loss: 0.2965 - accuracy: 0.9574 - val_loss: 0.3166 - val_accuracy: 0.9787
Epoch 19/100
21/21 [==============================] - 0s 3ms/step - loss: 0.2504 - accuracy: 0.9742 - val_loss: 0.3093 - val_accuracy: 0.9539
Epoch 20/100
21/21 [==============================] - 0s 3ms/step - loss: 0.2477 - accuracy: 0.9711 - val_loss: 0.2629 - val_accuracy: 0.9716
Epoch 21/100
21/21 [==============================] - 0s 3ms/step - loss: 0.2201 - accuracy: 0.9757 - val_loss: 0.2399 - val_accuracy: 0.9681
Epoch 22/100
21/21 [==============================] - 0s 3ms/step - loss: 0.1990 - accuracy: 0.9772 - val_loss: 0.2013 - val_accuracy: 0.9823
Epoch 23/100
21/21 [==============================] - 0s 4ms/step - loss: 0.1739 - accuracy: 0.9833 - val_loss: 0.2013 - val_accuracy: 0.9823
Epoch 24/100
21/21 [==============================] - 0s 3ms/step - loss: 0.1686 - accuracy: 0.9787 - val_loss: 0.1869 - val_accuracy: 0.9823
Epoch 25/100
21/21 [==============================] - 0s 3ms/step - loss: 0.1669 - accuracy: 0.9802 - val_loss: 0.1896 - val_accuracy: 0.9752
Epoch 26/100
21/21 [==============================] - 0s 3ms/step - loss: 0.1558 - accuracy: 0.9787 - val_loss: 0.1855 - val_accuracy: 0.9787
Epoch 27/100
21/21 [==============================] - 0s 2ms/step - loss: 0.1309 - accuracy: 0.9894 - val_loss: 0.1637 - val_accuracy: 0.9787
Epoch 28/100
21/21 [==============================] - 0s 2ms/step - loss: 0.1442 - accuracy: 0.9787 - val_loss: 0.1479 - val_accuracy: 0.9858
Epoch 29/100
21/21 [==============================] - 0s 3ms/step - loss: 0.1181 - accuracy: 0.9802 - val_loss: 0.1650 - val_accuracy: 0.9610
Epoch 30/100
21/21 [==============================] - 0s 2ms/step - loss: 0.1254 - accuracy: 0.9833 - val_loss: 0.1434 - val_accuracy: 0.9823
Epoch 31/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0973 - accuracy: 0.9954 - val_loss: 0.1188 - val_accuracy: 0.9823
Epoch 32/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0927 - accuracy: 0.9863 - val_loss: 0.1099 - val_accuracy: 0.9894
Epoch 33/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0884 - accuracy: 0.9909 - val_loss: 0.1219 - val_accuracy: 0.9858
Epoch 34/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0869 - accuracy: 0.9939 - val_loss: 0.1220 - val_accuracy: 0.9787
Epoch 35/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0953 - accuracy: 0.9818 - val_loss: 0.1143 - val_accuracy: 0.9823
Epoch 36/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0864 - accuracy: 0.9878 - val_loss: 0.1117 - val_accuracy: 0.9787
Epoch 37/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0734 - accuracy: 0.9909 - val_loss: 0.0915 - val_accuracy: 0.9787
Epoch 38/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0882 - accuracy: 0.9833 - val_loss: 0.0952 - val_accuracy: 0.9929
Epoch 39/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0718 - accuracy: 0.9909 - val_loss: 0.1084 - val_accuracy: 0.9787
Epoch 40/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0608 - accuracy: 0.9970 - val_loss: 0.0790 - val_accuracy: 0.9929
Epoch 41/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0628 - accuracy: 0.9909 - val_loss: 0.0824 - val_accuracy: 0.9929
Epoch 42/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0574 - accuracy: 0.9954 - val_loss: 0.0728 - val_accuracy: 0.9894
Epoch 43/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0507 - accuracy: 0.9970 - val_loss: 0.0751 - val_accuracy: 0.9929
Epoch 44/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0513 - accuracy: 0.9954 - val_loss: 0.0699 - val_accuracy: 0.9929
Epoch 45/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0448 - accuracy: 0.9970 - val_loss: 0.0792 - val_accuracy: 0.9787
Epoch 46/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0460 - accuracy: 0.9939 - val_loss: 0.0759 - val_accuracy: 0.9894
Epoch 47/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0425 - accuracy: 0.9954 - val_loss: 0.0676 - val_accuracy: 0.9894
Epoch 48/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0426 - accuracy: 0.9970 - val_loss: 0.0870 - val_accuracy: 0.9787
Epoch 49/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0388 - accuracy: 0.9985 - val_loss: 0.0617 - val_accuracy: 0.9929
Epoch 50/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0408 - accuracy: 0.9970 - val_loss: 0.0560 - val_accuracy: 0.9929
Epoch 51/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0340 - accuracy: 0.9970 - val_loss: 0.0560 - val_accuracy: 0.9929
Epoch 52/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0370 - accuracy: 0.9954 - val_loss: 0.0575 - val_accuracy: 0.9929
Epoch 53/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0340 - accuracy: 0.9970 - val_loss: 0.0551 - val_accuracy: 0.9929
Epoch 54/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0322 - accuracy: 0.9970 - val_loss: 0.0571 - val_accuracy: 0.9929
Epoch 55/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0337 - accuracy: 0.9985 - val_loss: 0.0547 - val_accuracy: 0.9929
Epoch 56/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0291 - accuracy: 0.9970 - val_loss: 0.0535 - val_accuracy: 0.9894
Epoch 57/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0289 - accuracy: 0.9985 - val_loss: 0.0473 - val_accuracy: 0.9929
Epoch 58/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0286 - accuracy: 0.9970 - val_loss: 0.0701 - val_accuracy: 0.9787
Epoch 59/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0294 - accuracy: 0.9985 - val_loss: 0.0685 - val_accuracy: 0.9823
Epoch 60/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0266 - accuracy: 0.9985 - val_loss: 0.0484 - val_accuracy: 0.9929
Epoch 61/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0242 - accuracy: 1.0000 - val_loss: 0.0466 - val_accuracy: 0.9929
Epoch 62/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0239 - accuracy: 0.9985 - val_loss: 0.0472 - val_accuracy: 0.9929
Epoch 63/100
21/21 [==============================] - 0s 5ms/step - loss: 0.0211 - accuracy: 0.9985 - val_loss: 0.0474 - val_accuracy: 0.9929
Epoch 64/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0202 - accuracy: 1.0000 - val_loss: 0.0441 - val_accuracy: 0.9929
Epoch 65/100
21/21 [==============================] - 0s 5ms/step - loss: 0.0204 - accuracy: 0.9985 - val_loss: 0.0443 - val_accuracy: 0.9929
Epoch 66/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0185 - accuracy: 1.0000 - val_loss: 0.0629 - val_accuracy: 0.9787
Epoch 67/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0222 - accuracy: 0.9985 - val_loss: 0.0458 - val_accuracy: 0.9929
Epoch 68/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0214 - accuracy: 1.0000 - val_loss: 0.0419 - val_accuracy: 0.9929
Epoch 69/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0181 - accuracy: 1.0000 - val_loss: 0.0539 - val_accuracy: 0.9858
Epoch 70/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0181 - accuracy: 0.9985 - val_loss: 0.0480 - val_accuracy: 0.9929
Epoch 71/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0174 - accuracy: 0.9985 - val_loss: 0.0417 - val_accuracy: 0.9894
Epoch 72/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0172 - accuracy: 0.9985 - val_loss: 0.0436 - val_accuracy: 0.9929
Epoch 73/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0179 - accuracy: 1.0000 - val_loss: 0.0730 - val_accuracy: 0.9823
Epoch 74/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0195 - accuracy: 0.9985 - val_loss: 0.0398 - val_accuracy: 0.9929
Epoch 75/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0158 - accuracy: 0.9985 - val_loss: 0.0390 - val_accuracy: 0.9929
Epoch 76/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0142 - accuracy: 1.0000 - val_loss: 0.0406 - val_accuracy: 0.9929
Epoch 77/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0128 - accuracy: 1.0000 - val_loss: 0.0403 - val_accuracy: 0.9929
Epoch 78/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0128 - accuracy: 1.0000 - val_loss: 0.0407 - val_accuracy: 0.9929
Epoch 79/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0124 - accuracy: 1.0000 - val_loss: 0.0392 - val_accuracy: 0.9929
Epoch 80/100
21/21 [==============================] - 0s 4ms/step - loss: 0.0122 - accuracy: 1.0000 - val_loss: 0.0380 - val_accuracy: 0.9929
Epoch 81/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0119 - accuracy: 1.0000 - val_loss: 0.0412 - val_accuracy: 0.9929
Epoch 82/100
21/21 [==============================] - 0s 5ms/step - loss: 0.0122 - accuracy: 1.0000 - val_loss: 0.0380 - val_accuracy: 0.9894
Epoch 83/100
21/21 [==============================] - 0s 5ms/step - loss: 0.0118 - accuracy: 1.0000 - val_loss: 0.0408 - val_accuracy: 0.9929
Epoch 84/100
21/21 [==============================] - 0s 5ms/step - loss: 0.0123 - accuracy: 0.9985 - val_loss: 0.0395 - val_accuracy: 0.9929
Epoch 85/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0116 - accuracy: 1.0000 - val_loss: 0.0345 - val_accuracy: 0.9929
Epoch 86/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0106 - accuracy: 1.0000 - val_loss: 0.0421 - val_accuracy: 0.9929
Epoch 87/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0103 - accuracy: 1.0000 - val_loss: 0.0356 - val_accuracy: 0.9929
Epoch 88/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0106 - accuracy: 1.0000 - val_loss: 0.0365 - val_accuracy: 0.9929
Epoch 89/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0107 - accuracy: 1.0000 - val_loss: 0.0379 - val_accuracy: 0.9929
Epoch 90/100
21/21 [==============================] - 0s 2ms/step - loss: 0.0092 - accuracy: 1.0000 - val_loss: 0.0385 - val_accuracy: 0.9894
Epoch 91/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0094 - accuracy: 1.0000 - val_loss: 0.0422 - val_accuracy: 0.9894
Epoch 92/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0093 - accuracy: 1.0000 - val_loss: 0.0365 - val_accuracy: 0.9929
Epoch 93/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0087 - accuracy: 1.0000 - val_loss: 0.0349 - val_accuracy: 0.9929
Epoch 94/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0081 - accuracy: 1.0000 - val_loss: 0.0366 - val_accuracy: 0.9929
Epoch 95/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0089 - accuracy: 1.0000 - val_loss: 0.0417 - val_accuracy: 0.9894
Epoch 96/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0085 - accuracy: 1.0000 - val_loss: 0.0383 - val_accuracy: 0.9929
Epoch 97/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0077 - accuracy: 1.0000 - val_loss: 0.0372 - val_accuracy: 0.9929
Epoch 98/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0075 - accuracy: 1.0000 - val_loss: 0.0359 - val_accuracy: 0.9929
Epoch 99/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0072 - accuracy: 1.0000 - val_loss: 0.0372 - val_accuracy: 0.9929
Epoch 100/100
21/21 [==============================] - 0s 3ms/step - loss: 0.0084 - accuracy: 1.0000 - val_loss: 0.0385 - val_accuracy: 0.9929
In [107]:
y_pred_1_3 = model_1_3.predict(X_test_tf)

9/9 [==============================] - 0s 0s/step
In [108]:
model_1_3.evaluate(X_test_tf, y_test_tf)

9/9 [==============================] - 0s 2ms/step - loss: 0.0385 - accuracy: 0.9929
Out[108]:
[0.03845573216676712, 0.9929078221321106]
In [109]:
plot_loss_curves(history_1_3)
In [110]:
accuracy_score(y_test_tf, y_pred_1_1.argmax(axis=1))
Out[110]:
0.9964539007092199
In [111]:
accuracy_score(y_test_tf, y_pred_1_2.argmax(axis=1))
Out[111]:
0.9929078014184397
In [112]:
accuracy_score(y_test_tf, y_pred_1_3.argmax(axis=1))
Out[112]:
0.9929078014184397

It look likes that changing learing rate doesn't affect performance. Maybe because we have very less data¶

In [ ]: